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I. INTRODUCTION 


The general aim of the research under this grant has been the joint 
optimization of the coding and modulation systems employed in telemetry 
systems. Particular emphasis has been placed on that research which would 
assist in the formulation of the inner and outer coding standards by the 
Goddard Space Flight Center. 

Section II provides a brief description of the major research achieve- 
ments during the first year of this grant, with reference to the technical 
reports or other publications where a -detailed description may be found. 
Section III lists the personnel who have been involved in this research. 

An appendix gives a detailed summary of the research done in concatenated 
coding systems. 

II.. SUMMARY OF RESEARCH PERFORMED 

A. Convolutional Code Construction 

Because of the desirability of standardizing as soon as possible 
upon a few convolutional codes, either long-constraint-length codes for use 
with sequential decoding or short- constraint-length codes for use with 
Viterbi decoding, it has become imperative to ensure that presently-known 
codes are optimum, or sufficiently close to optimum, so that a marked 
improvement would not later be forthcoming. 

Besides the traditional parameter of free distance, d^, as a measure 
of code goodness, research [1] under the predecessor of this grant has shown 

t. 

the importance of the distance profile, particularly for long-constraint- 
length codes used with sequential decoding. Moreover, as pointed out in our 
semi-annual status report [2] , there are situations in which long constraint 
length systematic codes might be preferable over the non-systematic codes 
now routinely employed. Thus, it became important to find optimum long- 



constraint-length systematic codes. This task was carried to completion by 
R. Johannesson. In his report on this research [3], convolutional - codes 
with an optimum distance profile are given at rate 1/2 for all constraint 
lengths K ^ 61. In' the same report, Johannesson also lists quick-look-in 
rate 1/2 codes ( 62 ( 0 ) = D + G^(D)) with an optimum distance profile for 
K ^ 51. This report also describes simulations which confirmed the principle 
investigator’s conjecture [ 2 ] that systematic codes perform equally as well 
as non-systematic codes under sequential decoding when the dummy information 
zeroes are suppressed in the tail of the systematic code. 

With the ever-increasing demand for greater reliability in decoded 
data, there has been a resurgence of. interest in concatenated coding. A 
major effort under this grant has been made to find convolutional codes that 
are optimum, or nearly so‘, for use with Viterbi decoding -in the inner coding 
portion of a concatenated coding system. The concern for the interaction 
of the various components in the total coding/modulation system led to the 
discovery by L. Lee [4] of a new type of convolutional code, the tmit-memory 
code, which is ideal for such inner system usage because of its "byte-oriented" 
structure as' opposed to the "bit-oriented" structure of cpnventional con- 
volutional codes. Lee found optimum unit-memory codes for all rates and 
constraint lengths of practical interest. We consider the discovery and 
development of this new type of byte-oriented convolutional code to be one 
of the principal achievements of this research and one certain to give in^ 
creased impetus to the use of convolutional codes in concatenated coding 
systems. 

In conjunction with the search for good long convolutional codes, 
simulations of sequential decoding on the deep-space channel have been carried 
out to compare directly .various convolutional codes that have been proposed 
as candidates for use in various deep-space systems. Detailed comparisons 



of rate 1/2, K = 32 codes have been conducted and reported by the principal 
investigator [5]. These simulations support the choice of the so-called 
"Massey-Costello" K = 32 quick-look-in code which is made in the proposed 
Goddard coding standard. 

B, Soft-Decision Demodulation 

In earlier work [6], the principal investigator demonstrated the 
value of the "cut-off rate" R of the discrete channel created by the 
modulation system as a measure of the quality of the modulation system for 
use with coding, and he showed how to design an optimum soft-decision demodu- 
lator for this criterion when binaiy signalling is used. An important advance 
made under this grant by Lee is the extension of all these results to lion- 
binary signalling [7]. Lee's work shows that the optimum soft-decision regions 
in likelihood space are always bounded by hyperplanes. Lee gave an algorithm 
for the determination of these optimal regions, as well as some heuristic 
rules for finding good, but sub-optimum, decision regions directly in signal 
space. The combination of the results in [6] and [7] provide a sound basis 
for the design of modulation systems to be used in conjunction with coding. 

C. Syndrome Source Coding 

• Some of the research under this grant has been concerned with the 
use of error-correcting codes to perform source coding or "data compressidn." 
Continuing his earlier work on syndrome source’ coding [8], Ancheta has during 
the past year made an important Innovation which he calls "noiseless universal 
syndrome source' coding" (NUSSC) and has demonstrated its robustness in com- 
pressing a broad range of sources [9]. Like its parent, NUSSC employs a 
very simple source encoder and hence appears very attractive for use on 
board spacecraft; the more complex source decoder being at the ground site. 
Ancheta' s innovation consists of using several different parity-check matrices 
and adaptively choosing the one to use in forming the syndrome according to 



the particular source output sequence. By cleverly choosing the parity- 
check matsdLces to be those of a nested sequence of error— correcting codes , 
Ancheta’s NUSSG scheme is almost as simple to implement as plain syndrome 
source coding. NUSSG appears to be a very practical method of data com- 
pression, and it is currently being applied to real telemetry data supplied 
by the Goddard Space Flight Genter to confim its effectiveness. 

D. Concatenated Coding Systems 

During this year of research, L. Lee has completed a major stixdy 
of concatenated coding systems which employ convolutional codes in the inner 
coding subsystem. This work forms the subject of Lee's doctoral dissertation 
[10] which is Included as the appendix to this report. Lee describes an 
almost bewildering array of options that are available to the designer of 
a concatenated coding system and performs the valuable service' of specifying 
the precise gain (in db) which each such option affords. The most sophisti- 
cated systems considered by Lee outperform all previous concatenated coding 

systems and represent nearly the ultimate in performance. It is expected 

\ 

that Lee's work will be the standard in this field for many years to come. 
III. PERSONNEL 

The table below lists all personnel who have been involved in the re- 
search tinder the first year of this grant. 

We are pleased to report that Mr. Lln-nan Lee completed the requirements 
for the Ph.D. degree in electrical engineering under this grant in November 
1975 and is now a member of the research staff of the Linkabit Gorporation 
in San Diego, Galifornia. 
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ABSTRACT 


of 

"CONCATENATED CODING SYSTMES 'EMPLOYING UNIT-MEMORY CONVOLUTIONAL 
CODES AND SYMBOL-ORIENTED OPTIMAL DECODING ALGORITHMS" 

by 

Lin-nan Lee. 

To achieve very reliable comitiunications in a very noisy 
channel with relatively small coding complexity, concatenated co- 
ding systems utilizing convolutional codes as the inner code and 
the Reed-Solomon (RS) codes as the outer code have been proposed 
by previous investigators. However, there has always been a 
"matching" problem between the bit-oriented convolutional inner 
codes and the byte-oriented RS outer codes. To use efficiently 
the potential of concatenation, we propose, in this dissertation, 
to concatenate a byte-oriented unit-memory convolutional code 
which has greater free distance than previously known convo- 
lutional codes of’ the same rate and the same state-complexity 
with RS -outer codes of the same symbol size. We also propose 
to utilize a Real-Time Minimal-Byte-Error Probability (RTMBEP) 
decoding algorithm in conjunction with the feedback from the 
outer decoder as the decoder for the inner convolutional code. 

The performance of this concatenated coding system is studied, 
and the improvement due to each additional feature is calculted. 

It is shown by simulation that this concatenated coding system 
out-performs all previously known concatenated coding schemes. 
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CHAPTER I 


INTRODUCTION 

Shannon's celebrated theory of information states 
that information can be transmitted with an arbitrarily small 
error probability provided that the rate of transmission is below 
channel capacity. Communications, engineers now recognize that 
this promise of highly reliable communications can be achieve only 
by means of coding. 

Because the complexity of coded communication systems 
grows exponentially with the block length for block codes (or 
with the constraint length for convolutional codes) , instead of 
directly using very long codes, the idea of cascading two or more 
codes of less complexity to achieve higly reliable communications 
was first considered by Elias [1] , and later by Forney [2] . 

Forney's, technique of using two or more block codes over different 
alphabets to obtain very low error rate, over noisy channels is 
known as concatenated coding. Guided by the premise that a con- 
volutional code generally, performs better than a block code of 
the same complexity. Falconer [3], and later Jelinek and Cocke [4] 
tried to cascade block codes and convolutional codes. Figure 1.1 
shows a general representation of a block-convolutional concatenated 
coding system. 

In their schemes, sequential decoding is used for the inner 
decoder. However, the performance of sequential decoding is such 
that the probability of error can be reduced very sharply with 
slight increment o.f signal energy if >the rate of transmission is 


1 



2 
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below the computational cut-off rate, ^comp' primary function 

of the outer block code is only to resolve decisions when the 
sequential decoder experiences computational overflow. Therefore, 
the over-all coding system can be regarded, more or less, as a 
sequential decoded convolutional coding system. 

The implementation and operation of Viterbi decoding 
are later found simpler than sequential decoding in a number of 
respects. In particular, with a moderate constraint length con- 
volutional code, Viterbi decoders can be operated with an error 

-2 

rate less than 10 at a rate slightly higher than From 

Forney's result [2]., it is conceivable that a concatenated coding 

system with a powerful outer code can perform reasonably well 

when its inner decoder is operated with a probability of error in 

-2 —3 

the range between 10 and 10 . Odenwalder [5] , then, chose a 

Viterbi decoder for the inner coding system. We recall that it is 
always possible to achieve very low error probability by means of 
sequential decoding of long constraint length convolutional codes 
at rates slightly below the "cut-off rate", R (R ) , of the 
channel if one can tolderate a certain amount of erasures. There- 
fore, in order to be an attractive alternative to sequential decoding, 
a concatenated coding system has to operate at an overall rate 
slightly above the cutpoff rate of the channel. On the other hand, 

we can always regard the convolutional encoder-discrete channel- 

1 

decoder combination in Fdgure 1.1 as a "super-channel" for. the 
outer coding system; the function of the inner convolutional coding 
system is then to create a better channel for the outer coding system. 
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Hence it is necessary that the inner convolutional coding system 
be operated at a rate slightly below the channel capacity so that 
the super-channel is at least slightly "cleaner" than the original 
discrete channel. This requirement has thus limited the choices 
of outer codes to those of very high rate. And because it is usually 
difficult to implement Viterbi decoders for high rate convolutional 
codes with large enough free distance, high rate block codes are 
the only option left for the outer coding system. 

Generally, we can reduce the total signal energy per 
information bit required to achieve a certain error probability for 
a given channel by employing a lower-rate code and thus spending less’ 

I 

energy in each baud. For example, simulation result in section 5.4 
shows a 0.5 dB improvement in the signal-energy-per-information-bit 
to one-sided noise-power- spectral-density ratio when one elects to 
use a rate 1/3 convolutional coding system of the same decoder com- 
plexity in place of a rate 1/2 convolutional coding system. But 

for a given data rate, the lower rate coding system implies a higher 

* 

bandwidth, which in turn implies the need for a much more sophisti- 

f . 

Gated signal detection device. The difficulty in operating the phase 

locked loop in the detection device increases with the bandwidth and 
negates the advantage of using very low rate, codes. . Experience 
has shown that convolutional codes of rate 1/2 and 1/3 are the most 
attactive candidates for the inner coding system. Furthermore, 
because the complexity of Viterbi-like decoders for convolutional 
codes is exponentially dependent on the constraint length of the 
convolutional code, the task of finding "good" inner codes for the 
concatenated coding system is then focused to the search for rate 
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1/2 and rate 1/3 convolutional codes of short constraint length ' 
which are capable of good performance on a very noise channel. 

Since the output error patterns of Viterbi-type decoders 
for convolutional codes are usually bursty, block codes over a large 
alphabet, such that many bits of the inner code form one symbol of 
the outer code, appear very attractive for the outer coding system. 
In particular, it appears that the Reed-Solomon (RS) block codes 
are the most satisfactory because there are relatively simple de- 
coding procedures (such as the Berlekamp-Massey [6] , [7] , algorithm) 

for RS codes and because of the "maximum-distance- separable" pro- 
perty of the RS. codes. But the lengths of the bursts of errors 
made by Viterbi-like decoders are widely distributed so that it is 
generally necessary to interleave the decoder output for the inner 
convolutional code such that errors in the individual RS-symbols of 
one block are independent. Otherwise, we would have to use a very 
long block code to operate the system efficiently. 

From the above general discussion, we conclude that 
Odenwalder ' s configuration of a block-convolutional concatenated 
coding system is generally a sound choise. However, we have ob- 
served that we may further improve the performance of this type 
of concatenated coding system in several different directions. 

These possible directions of improvement will be studied in the 
following chapters of this dissertation. . 

Because the most-likely burst length of the decoding 
error patterens made by the inner decoder are on the order of the 
'Gonstv-aint length of the inner .convo-lutionai oode* Odenwalder chose 
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the RS symbol to be of the same size as the constraint length of the 
convolutional code. Although this is certainly a reasonable choice, 
it can be improved upon. For example, it is very unlikely that the 
beginning of a burst is always aligned with the boundary between two 
RS symbols; therefore, it is possible that a burst only two bits long 
may affect two RS symbols. This fact leads us to the idea of 
constructing good convolutional codes which are symbol-oriented 
rather than bit-oriented. In Chapter II, we discuss a systematic 
approach to constructing such code, and we shall see that the codes 
thus constructed generally have free distance better than Odenwalder's 
convolutional codes of roughly the same complexity in terms of the 
Viterbi decoder implementation. In fact, because of this improved 
free distance and the symbol oriented nature of these codes, we 
obtain an approximately 0.3 dB improvement in the over-all per- 
formance of the concatenated coding system when these. codes replaces 
Odenwalder ' s codes . 

Another possible improvement is to modify the decoder for 
the convolutional code -such that the decoder emits not only the most- 
likely estimated symbol, but also reliability information about the 

i 

estimated symbol. Based on this reliability information, the outer 
decoder is then able to perform either ''erasures-and-errors*' decoding 
or "generalized-minimum-distance". (GMD) decoding as suggested by 
Forney [ 2 ]. Zeoli [8] and Jelinek [9] have proposed. methods of 
extracting reliability information from a Viterbi decoder. Concep- 
tually, their approach is to annex a long tail to the original 
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convolutional code and to use this added tail to provde an error de- 
tection capability for the estimate made by the Viterbi decoder for 
the original shorter convolutional code. This approach requires , 
feedback from previously decoded symbols in the Viterbi decoder 
and, more importantly, uses the symbol as corrected by the outer 
decoder to restart the inner Viterbi decoder whenever an error is 
corrected by the outer decoder. We find that this feedback from 
the outer decoder improves the performance by 0.3 dB, while the er- 
ror detecting capability and the "erasures-and-errors" decoder 
provide an additional improvement of 0.2 dB. 

We have also studied algorithms v;hich compute the a 

f 

posteriori probability of each decoded symbol for the short con- 
strain!: length convolutional code and which use this a posteriori 
probability as the reliability information provided to the outer 
coding system. Although this technique proved to be less powerful 
than Zeoli's tail annnexation scheme (this erasure scheme improves 
the performance by only 0.05 dB to 0.1 dB over hard-decision deco- 
ding) , its performance is undoubtly optimal among all the possible 
schemes employing only a short constraint length convolutional 
code without an annexed tail, because decoding decisions are based 
on the a posteriori probability calculated. Moreover, in conjection 
with the use of the feedback, the a posteriori probability decoding 
algorithms seem to perform much better than the Viterbi decoder does 
when aided by feedback from the outer decoder (approximately 0.2 dB 
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difference) . The algorithm used together v;ith feedback from the 
outer decoder, even without the extra tail, offers a slight im- 
provement over Zeoli’s scheme. Moreover, in this scheme, the inner 
encoder and the inner decoder have the same constraint length 
and the decoder can return to normal operation a few branches after 
•an error event occurs. It is possible to interleave the output of 
the decoder for the convolutional code to create a memory less super- 
channel for the RS decoder. This results a simpler implementation 
than that for Zeoli's type of system. These algorithms, which • 
we call the Real-Time Minimal-Byte-Error (RTMBEP) decoding algorithm, 
and the Real-Time Minimal -Bit-Error Probability (RTMbEP) decoding 
algorithm are described in detail in Chapter III. 

Another area of possible improvement which one can 
visualize for Odenwalder’s system is the area of soft-decision’ de- 
modulation. Using the criterion proposed by Wozencraft and Kennedy 
[10] and by Massey [11] we can "optimize" the demodulator by maxi- 
mizing the cut-off rate, R^, of the resultant discrete channel. 

In chapter IV, we shall show that the decision boundary of the 
demodulator maximizing R^ are hyperplanes in likelihood space. 
Although we are not able to improve the performance of coded com- 
munication system very much in the 'case of binary signaling with 
this optimal demqdtilator because Viterbi decoding is relatively 
insensitive to the demodulator design, we believe that the decision 
rules for optimal demodulators thus obtained may be useful for 
future soft-decision decoding, because, the symbol-oriented decoding 
algorithms described in Chapter .III compute the reliability in- 
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formation not only for the most-likely symbol but also for all 
symbols in, the coding alphabet and, therefore, make available the 
likelihood-ratio vector. Although this aspect of the study has 
not been carried very far and deviates somewhat from the main 
line of our research, we summarize in Chapter IV some scattered 
results which are related to the subject of this dissertation. 

In Chapter V, we give the performance of various types 
of concatenated coding systems as obtained from simulations and 
compare the improvement in performance due to each feature employed 
in the system. Finally, the results and conclusions are summarized 
in Chapter VI. 



CHAPTER II 


BYTE-ORIENTED CONVOLUTIONAL CODE 

In this chapter, we introduce "unit-memory" convolutional 
codes which are "byte-oriented" in such a way as to be attractive 
for use in concatenated systems. We shall show that con- 

volutional codes with unit memory always achieve the largest free 
distance ■ among all codes of the same rate the same ‘number 

2^0 of encoder states, where M is the encoder memory. 

2. 1 BINARY CONVOLUTIONAL CODES 

Let the binary -tuple denote the subblock of 

information digits at time t (t =0, 1, 2, ...), and let the binary 

n - tuple b^ denote the encoded subblock at time t in an (n , k ) 
o — t o o 

convolutional code. Then, the encoding equations may be written 

b^ = Gg + + . . . + (t = 0,1,2,...) (2.1) 

where each G. is a k x n binary matrix, where M is the code me- 
mory, where the operations are in GF(2), and where, by way of con- 
vention, a^ = 0. for t>0. An encoding circuit is shown in Figure 

MK 

2.1. Note that the encoder has 2 o distinct states where the state 
is taken as the contents of the delay cells in the encoder. We shall 
refer to the number Mk^ of binary state variables in the encoder as 
the state-complexity of the convolutional code. 

The constraint length K {measured in information digits) of 

the convolutional code is defined by 

K - (M + 1) k . 

o 

* This chapter of the dissertation is taken from [28] . 
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The rate of the code is defined by 


R= k /n . 
o o 

In virtually all past applications of convolutional codes, and 
have been taken as relatively prime, i.e., gcd (k^, n^) = 1 where 
"gcd" denoted "greatest common divisor". In fact, the condition 
gcd (k^,n^) = 1 is generally tacitly assumed so that speaking, 
for instance, of a convolutional code as being of rate R = 1/2 
would imply k^= 1 and n^^.2 unless the contrary were explicitly 
states. As will be seen, however, there can be advantages in 
taking gcd (k^,n^) >1, 

For convenience, let b . . ^denote the encoded sequence 

. :b^ , ] over time units t through t’ and let de- 

note the entire semi- infinite encoded sequence. Let aj-^ and 
a^-Q j be similarly defined. The free distance , of the 

convolutional code is the, minimum Hamming distance between all 
pairs of encoded sequences b^.^ ^ ^ resulting from pairs of inf or— 



igure 2,1. An encoding Circuit for an (n^, k ) convolutional 
code wich memory M. o o 







the linearity of (2.1), it follows that 


= min W (b , » . ) 

free -[0 ,c»)^ 


( 2 . 2 ) 


where W(*) denotes the Hamming weight of the enclosed binary se- 
quence and vrhere. the minimization is over all am such that 

— I u f ; 

code is non-catas trophic [12,, 13] when W(aj-Q “ 

implies that W(b._ ,)=>», in which case the minimization in (2.2) 

— I u /«» j 

reduces to the minimum over all ar ^ v such that a_?^ 0 and 

— L u , 00 j - — 0 

W(£^q ^j)=“ . The restriction to non-catas trophic codes entails 

no loss in the achievable value of d^ for given n , k and M, 

free o o 

a fortunate situation because the non-catastrophic property is 
essential in applications (cf . [12]) . 

Because d,^^^ is the primary determiner of decoding error 

probability when Viterbi (i.e., maximum likelihood) decoding is 
used with a non-catastrophic code, is the usual criterion 

of goodness for codes to be used v/ith Viterbi decoders. Because 

the number of states of the Viterbi decoder [14] coincides with 

Mk 

the- number of encoder states, viz, 2 o, practically dictates a 
small state-complexity. The region Mk^_< 6 appears to be about 
the range where Viterbi decoding is attractive in applications. 
Thus, for Viterbi decoding applications, we are motivated to find, 
for a given code rate and a given state-complexity in the above 
range, a convolutional code with maximum dg^^^. In the next 
section, we report the results of our search for such codes and 
we also derive a useful upper bound on the attainable 
2 . 2. UNIT-MEMORY CODES WITH MAXIMAL d 


free 


Any (n^,k^) convolutional code with memory M can be consi- 
dered as an (n^= Mn^, k^= Mk^) code with M*=l simply by taking 



13 


^0 = 


T 


^M-2 


and 


4 = 


0 

, M 

M-1 M 


lS 


0 

0 


M 


These two codes are entirely equivalent in the sense that the same 
semi-infinite binary information sequence produces the same semi- 
infinite binary encoded sequence, although the division into time 
subblock would be different. Since the state-complexity is Mk^ 
for both codes, it follows that for a given state-complexity and 
given rate, the maximum value of ^ is achieved within the sub- 
set of convolutional codes with M = 1, Hence, we can restrict our 

search for optimal codes, for a given rate and state-complexity, 
to codes with unit-memory . 

For a unit-memory code, equation (2-1) reduces to 

(t = 0,1,2,. . (2.3) 

When _0 is the only non- zero information subblock, then 

-[0,1)"^ -0 (2.4) 

is the only possibly-non-zero portion of. b.,. ». From (2.2), it 

— [ U , 00 J 

then follows that the attainable d^^ of a unit-memory (n„,k_) 

’*"■ — - - - — ■ . jTjr©© * o o 

convolutional code is upper-bounded by the largest minimum dis - 
tance of an (n =2n^, k = k^) block code . We shall call this up- 
per-bound the block code upper bound on note that 

McEliece and Rumsey [15] have used similar arguments to derive more 
elaborate upper bounds on 


■^t " '^t^O ^t-l®l 


for codes where M ^ 1. 
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We see that the argument used above to establish the 
block code upper bound on suggests the following search 

procedure for finding a non-catastrophic unit-memory (n^, k^) 


code with maximal d^ : ' 

free 

(i) Set d equal to the largest achievable by any 

(n= 2 n^, k = k^) block code. 

(ii) Choose [GqiG^^] as the generator matrix of a (nF2n^, 

k = k ) block code with.d . = d. If d^ • ~ d and the code 

o min free 

is non-catastrophic, stop. Otherwise continue with step (ii) 

until all block codes with d . = d have been exhausted. 

min 

(iii) Reduce d by 1 and return to step (ii) , 

The above search procedure was carried out to obtain, 
for rates 1/4, 1/3, 1/2 and 2/3 (which are the usual rates of 
interest in applications) , a non-catastrophic unit-memory con- 
volutional code with maximal for all state-complexities 

of 6 or less. The values of obtained are given in 

Table i where we- also list, for comparison, the largest 

attainable by a code of the same state-complexity having 
gcd (n^, k^) = 1. The block code upper bound on 
each case is also listed. 


The codes with gcd (n , k ) = 1 that achieve the values 
of given in Table 2.1 may be found in Larsen [16] and 

Paaske [ 17 ] The values of the block code upper bound on 
given in Table I, were taken from Calabi and Myrvaagnes [18] 
and from Helgert and Stinaff [19]. In Table 2.2 we give the 
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matrices Gq and of a non-catastrophic unit-memory code 
with maximal at each place where Table 2.1 shows that 

value of exceed the available from the best gcd (n^, 

code having the same state-complexity. 


Rate 

State Complexity 

Block Code Upper 

Maximal of 

linit-Memory^5odes 

Maximal 

0 0 

(Ho. of State Variables) 

Bound on d_ 

free 

gcd (n 
Codes 

1/4 

1 

8 

7 

7 


2 

10 

10 

10 


3 

13 

13 

13 


4 

16 

16 

16 


5 

20 

20 

18 


6 

24 

24 

20 

1/3 

1 

6 

5 

5 


2 

8 

8 

8 


3 

10 

10 

10 


4 

12 

12 

12 


5 

15 

15 

13 


6 

16 

16 

15 

1/2 

1 

4 

3 

3 


2 

5 

5 

5 


3 

6 

6 

6 


4 

8 

8 

. . 7 


5 

9 

9 

8 


6 

10 

10 

10 

2 2/3 

2 

■ 4 

3 

3 


4 

6 

6 

5 


6 

8 

7 

7 


Table 2,1: Maxinal for a given state-complexity Mk^ of 

unit memory convolutional codes and of convolutional 

codes with gcd (n , k ) = 1. ; 

0 0 


of 

r= 1 
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Rate n k 
o o 


Go 


Gi 


1/2 8 4 10000111 

01001011 
00101101 
00011110 


10001011 

11100010 

10111000 

11010001 


1/2 10 5 1000011111 

0100001111 
0010011110 
0001011001 
0000110101 


1111111111 

1111000000 

0010110100 

1010011010 

0110101001 


1/3 15 5 100001111011010 

010000111101101 
001001011110110 
000101101101011 
000011110110101 


000111110010100 
001101100101010 
011001001100101 ‘ 
110000011110010 
100010111001001 


1/3 18 6 111000110100110000 

011100011010011000 
001110001101001100 
000111100110000110 
100011010011000011 
110001101001100001 


OOOOllOOOlllOOlOll 

000110001110010110 

001100011100101100 

OllOOOlllOOOOllOOl 

llOOOOllOOOlllOOlO 

100001100011100101 


1/4 20 5 10000011111110011000 

01000101110111001100 
00100110110011100110 
00010111011001100011 
00001111101100110001 


00011110100001100111 

00110011010110001110 

01100101101000111100 

11000010110011011001 

10001101011100010011 


1/4 ' 24 


100000111110111010110000 

010000011111011101011000 

001000101111101110001100 

000100110111010111000110 

000010111011101011000011 

000001111101110101100001 


001111000101001011010011 

011110001010010110100110 

111100010100101100001101 

111001101000011001011010 

110011010001110010110100 

100111100010100101101001 


2/3 6 4. 100001. 

010010 

001011 

000111 


111100 

110110 

010011 

101001 


d 


free 


8 


9 


15 . 


16 


20 


24 


6 


Table 2.2, The encoding matrices of- some non-catas trophic unit-memory 

convolutional codes with free distance greater than the 

maximal free distance of ged (n , k ) = 1 codes of the same 

o o 

rate and state complexity. 
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2.3. BYTE -ORIENTED NATURE OF UNIT-MEMORY CODES 

■*V 


We now show that short, unit-memory convolutional codes 
are "byte-oriented" in such a way as to be attractive for use, 
with Viterbi decoding, as the inner coding component of a con- 
catenated coding system. 

In general , the state at time t of the convolutional 
encoder is the .information sequence t-1] ~ ^-t-M*— t-M+1* 

. , . :a^_j^] over the preceding M time units. Note -that the suc- 
cessor of this state, namely, * •* * 'ft, t-i*— t^ ' already 

k 

determined up to the 2 choices of a. , i.e., each state will 
k© 

have 2 successors in the "trellis" defined by the convolutional 

code [12] . In the corresponding Viterbi decoder, the "metric" 

Mk 

for the best path to each of the 2 ° possible states at time 


k 

t must be relayed to each of its 2 ° successors. Hence, the 
value of kQ influences the overall complexity of the Viterbi 
decoder, although much less strongly then does the state-com- 
plexity. Nonetheless, to determine, for instance, whether the 
Viterbi decoder for an R = 1/3, M=^l, k^ = 6 code (state-com- 
plexity 6) is simpler than the Viterbi decoder for an R-= -1/3, 

M = 7, = 1 code (state-complexity 7) would require a detailed 

analysis of the specific decoder design. 

With Viterbi decoding of convolutional codes, there is a 
natural segmentation of the decoded information digits into bytes 
of k^ bits, because the information byte a^ acts as a unit in 
determining the correct state. The typical decoding "error 
events " [13 ] give rise to the incorrect decoding of a small 
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number o£ bytes, viz., the average number of non-zero information 
bytes in an information sequence aj-^ that generates an encoded 
— [0 ~) Hamming weight In an M = 1 code, this average 

is near 1, although there will generally be about k^/2 bit errors 
per byte. In a code with = 1, there will generally be a small 
(say, about 3) of byte errors but, since a byte is a bit in this 
case, the same number of bit errors. However, if the information 
bits with the latter code are gathered into "bytes" of Mk^ = M 
bits, there will generally be about 1 "byte" errors per error 
event since the bit decoding errors are not synchronized to begin 
at the start of these "bytes." Thus, even if both codes have the 


same state-complexity and same one would expect' the unit- 

memory code to have a lower byte-error probability' for bytes 


of Mk^ bits. 


To test the validity of these observations, we simulated 


Viterbi decoding on an additive white Gaussian noise (AWGN) channel 


with several values of the energy per information bit to one-sided 


noise power spectral density ratio, for several convolu- 

tional codes of rate R = 1/3. The codes tested were (i) the k^= 6 
unit-memory code of Table 2.2, (ii) the ^ ~ 6 code given 

by Larsen [16J and (iii) the k^ = 1, M = 7 code given by Larsen 
Elgl. The byte size was 6 bits. The results of the simulation 


are given in Table 2.3. The unit-memory code (i) had a decoding 


byte-error probability about one-half that of the gcd k^) = 1 

code (ii) with- the same state-complexity (Mk^ = 6) . This super- 
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iority of code (i) over code (ii) is due mainly to its greater 
free distance, 16 as opposed to 15. But we also see from Table 
2.3 that the decoding' byte-error probability of code (i) is about 
two- thirds that of code (iii) which has the same free distance 16 
and greater state-complexity, 7 versus 6. This superiority of 
code (i) over, code (iii) is due entirely to its byte-oriented 
nature . 

We conclude that the unit-memory codes, because of their 

byte-oriented nature, appear attractive for use as the inner code 

in concatenated coding systems [2] where the outer code is a 

k 

Reed-Solomon (RS) code over the alphabet GF(2 , ,i.e. the bytes 
of the convolutional code are single digits for the RS code. 

For instance, code (i) above would be used with an RS code over 
GF(2 ). We shall report in detail on the effectiveness of unit- 
memory codes in concatenated coding systems applications in 
Chapter V. 


^b/^o 

M=1 (kS, 

6) code 

M-6, (3, 

1). code 

M=7 (3,1) 

code 

(dB) 

Byte- 
Error 
Prob. ' 

95% Con-, 
fide nee 

Byte- 

Error 

Prob. 

95% Con- 
fidence 

Byte- 

Error 

Prob. 

95% Con- 
fidence 

1.00 

0.03050 

+0.00533 

0.04875 

+0.00681 

0.04000 

+0.00619 

1.25 

0.02000 

+0.00435 

0.03250 

+0.00561 

0.02250 

+0.00469 

1,50 

0.01175 

+0.00329 

0.02325 

+0.00477 

0.01400 

+0.00372 

1.75 

0.00650 

+0.00250 

0.01275 

+0.00350 

0.01025 

+0.00319 


Byte-error probability for Viterbi decoding 
of three R=l/3 convolutional codes on a simulated 
AWGN channel . 


Table 2.3-: 



CHAPTER III 


BYTE-ORIENTED DECODING ALGORITHMS 
FOR CONVOLUTIONAL CODE* 

In the previous chapters/ the terminology "byte", or 
"symbol", has appeared in' several places to denote the basic 
grouping of digits such that thp distance measure between any 
two sequences is the niamber of different "bytes" or "symbols" 
between them. In other words, the byte is defined to be the 
single "super-symbol" for which the Hamming- distance measurement 
or the error rate, is concerned. The byte size is determined by 
the nature of the application. For example, 128 characters are 
included in the standard teletype alphabet, the byte size of 
tyletype signals is, then, 7 bits. In the case of concatenated 
coding, only the symbol error rate 'of the outer coding system 
can be controlled, the byte size is therefore the symbol size 
of the outer code. 

For convenience of discussion, we shall assume that the 
byte size is integer multiples of k^, and a byte covers m 
time instances. We shall denote an information byte encoded 
between time t and t+m-1 as 

^ ^t+l' “ * ^t+m-1^ ' 

V7here T = (t+m-1) /m is implicitly implied. Further, if the 
encoding shift-register is initially loaded with zeros, after 
vzhich ’ ' :^L encoded, the followed by Mk^ zeros, 

* Parts of this chapter are extracted from l23 ] .and [25 ]« 
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{i. e ,,, = 0} to clear the encoder, L is called 

the frame length. In a frame, we have L' = L/m bytes. The 
byte error probability of a frame 


^BE ^’t=1 ^ 

is the quantity used to measure the quality of information 
transmission. The Viterbi decoding algorithm [20] , which is 
the maximum likelihood decoding algorithm for a convolutionally 
coded frame sent over a discrete memoryless channel, forms as 

A 

its estimate A,, the information sequence that maximizes 

[ -L , Jj j 

the conditional probability. 


^^^[1,LM ^ -U,L+Ml^' 

based on the sequence r|.^ L+M] output of the 

discrete memoryless channel. Hence, this algorithm minimizes 
the frame error probability . 



P 


r 


^~[1,L‘] ^ -[1,L']^ 


for any interesting channel , it must- be true that 


lim 
L * 


FE 


1 , 


so that is not a meaningful optimality criterion for large 
frames. However, since we can write the byte-error-probability 
as 


P 


BE 


1 

L* 


L' 


I 

T=1 







it follows that 
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P < P < L 'P 
^BE - ^FE - ^ BE 


Hence, when L’ is fairly small, it makes little difference whether 
P__ or P__ is minimized (which explains the appropriateness of 
Viterbi decoding when L' is small) . The byte-probability, P„„, 
is minimized by the decoding rule which, for each T, 1^T£L', 
chooses its estimate as the byte which maximizes the conditional 
probability 


-[1,L+M] ^ * 

When the byte size is one bit, algorithms, similar to Viterbi *s 
to accomplish this maximization have been proposed independently 
by Bahl et ^ [21] , and McAdam ^ al [22] , If the byte size 
is ' smaller than or equal to the size of the memory M, but larger 
than a. bit, the natural extension of these algorithms making use 
of the state property of the convolutional codes is self-evident. 
However, these algorithms require receipt of the entire frame 
— [1 L+M] decoding begins and so cannot be used without 

resynchronization (i.e. cannot be used when L=”) . Moreover, 
their implementation requires storage which grows linearly in 
L and, hence are practical alternatives to Viterbi decoding 
only for moderately small L. 

In the following section we derived a recursive procedure 


(incorporating the observation of Pritchman and Mixsell [ 24 ] ) 

for "real-time minimum-bit-error probability (RTMbEP) decoding 

" for (n ,1) convolutional codes to minimize Pu,n under the cons- 
o bE 

traint that the decoding delay be limited to A branches. (Here 
we use the lower-case letter "b" to remind the reader that the 
byte size discussed is one bit.) In section 3.2 we formulate 
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the corresponding decoding algorithm and show that its storage 
requirements are independent of L. We also show the necessary 
modification needed to minimize the byte-error probability 
instead of the bit-error probability. This algorithm can be ge- 
neralized to decode any (n^/k^) convolutional code, but, because 
of the particular importance of unit-memory convolutional codes, 
we show a modified version of the RTMBEP decoding algorithm for 
(no,ko) unit-memory convolutional codes in section 3.3 which is 
much more efficient than that for general codes. For comparison 
purposes, we also formulate a "real-time", modified Viterbi 
decoding algorithm in section 3.4. Section 3-5 reports the results 
of using these decoding procedures on a simulated additive White 
Gaussian noise (AWGN) channel- It is concluded that the impro- 


vement in Pgg for the real-time minimal-byte-error probability de- 
coding (RTi-iBEP) algorithm is not enough to justify the added com- 
plexity compared to Viterbi decoding in- hard-decision applications 


but, as shown in the later chapters, the new algorithm offers ad- 
vanges in soft-decision applications such as concatenated coding. 


3.1 DERIVATION OF A REAL-TIME MINIMOM-BIT-ERROR PROBABILITY 
DECODING PROCEDURE FOR (n^/D CONVOLUTIONAL CODES 
As in all previous optimal {in some sense) decoding pro- 
cedures for convolutional codes , we shall make important use of 
the encoder state which at time t is defined as the contents of 
the shift-register in Fig. 2.1, i.e, the M- tuple of past informa- 
tion bit 

®t ~ ^^t-l'^t-2' * ’ *®-t-M^ 


(3.1) 
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and where by our convention, a^=0 for i^O and i>L. As the 
term "state" implies, completely accounts for the past history 
of the encoder input in the sense that and the input segment 
a uniquely determine the output segment b L+M] * condi- 

tioning of the encoder state, the calculation of the probabilities 
required for the decision rule can be simplified. 

The decoding rule which minimizes under the' "real-time" 

constraint that a, be decided from rn ^.,*1 is that which chooses 

t — Ll,t+AJ 


= 0 if 


(3.2) 


and chooses a^=l o-therwise (where we have arbitrarily resolved 

A 

ties in favor of the decision a^=0.) Since 




^^^^[l,t+A]^ 


and since the _ probabilities on the righ-hhand side of this latter 
equation can be expressed as the summation over all states of 
the joint probabilities including the state, we have 






I ^*-(l,t+A]'®t+4+l 

o 


(3.3) 


We now proceed to develop recusive formulas for the two prob- 
abilities appearing on the righthand side of (3.3). 

For any t, t>l, we may write 


^ ^-[l,t] ^®t+l^ ^ ^ ^~[l,t] "^t'®t+l^ ^ 


(3.4) 



but also 






(3.5) 


where we have made use of the state property and our assumption 
that the channel is memoryless. ■ Writing 




(3.6) 


we then use the fact seen from (3.1) that the state s 


t+1 


[a^,a^_j^, . . . has only two possible predecessors s^, 

namely ‘ 


^t+1 ' t 0 otherwise 


(3.7) 


where here and hereafter we write p(s) for the set containing 
the two possible predecessors of a state s. Finally, we 
note that, for and s^fand we write k^®t'^t+l^ for this 

branch ] so that 


and we note that this quantity is determined by the channel 
transition probabilities. Substituting (3. 5) , (3. 6) , (3. 7) and 
(3.- 8) into (3.4) we have our desired recursion 
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(3.9) 




s^ep(s^^l) 


The starting value ~ needed to apply .the 

recursion is simply 


P(r^,S2) ={ 


I P(rj^|b(0,S2)) 
0 


if OeP (S 2 ) 
otherwise 


(3.10) 


where we have used the fact that Sj^=0. 

By an entirely analogous argument whose details we omit, the 
following recursion for the other probability on the righthand 
side of (3.3) may be obtained: 


P (^t'-[l,t+i] '^t+i+1^ 

s 


^ 2 ^ ^-t+i^-^®t+i,®t+i+l^ ^ 

t^i£p 


P^^t^~[l,t+i-l] ^®t+i^ 


(3.11) 


which we shall use for M < i < A. The starting value needed for 
this recursion is 




^ ^^^[l,t3 ' ^^l£[l,t] ,®tW 


j ^^^[l,t]'^t+M^ ^t 


I 


otherwise j 


so that the quanities obtained from the recursion (3.9) directly 
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provide the necessary initial conditions (3.12) to be used with 

N 

the recursion (3.11). Hence we have now obtained a complete 
recursive procedure for performing real-time minimum-bit-error 
probability decoding of convolutional codes . 

‘ 3.2 IMPLEMENTATION THE DECODING ALGORITHM 

We now describe an algorithm for implementing the decoding 
. procedure whose recursive basis was developed in the previous 
section. Our algorithm requires the storage of two real numbers 
for each of the 2^ encoder states. We denote these stored quanti- 
ties for state s as f (s) and g(s) . At time t+M-1 (in the algorithm) 
the first of these quantities will store the value 


f(s) ^ ^£^[1, t+M-1] '®t+M ' 

and the second will store the value 


(3.14) 


h(s) . P(a^=0, 


(3.15) 


when j is indicated from M' through A within the time interval 
[t+m-1, t+mj in the algorithm. Vlhen j=A, we store the value of 
h(s) summed over all states which we denote as X^e keep the 

previous A -m values of this quantity and denote these stored values 
as H^, H 2 /..., that is, 

H^ = S P(a^_^=0, £[i^t+A-M-i] ' ^t+A-M-i-l-l'"®^ (3.16) 

for Ki£A-M will be the available value at time t+m-1 (in the 
algorithm) . It follows that and 


Sf (s) 
s 


^P(£[l,t]' ®t+l 


s) 


(3.17) 
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are the desired numerator and denominator on the right-hand 

side of (3.4) for the decoding of which is accomplished at 

time t+H-1 (in the algorithm) . The only additional storage 

required is that for the received branches r. r. ,r. . 

^ _ — t+M' — t+M+1 — t+A 

The recursions (3.10) and (3.14) directly correspond 
to the following: 

The RTMbEP Decoding Algorithm- 

Step 0 : Set t=l/ set fCs) = ^ P (r^ |b (0/S)) for the stored 

states s having 0 as a predecessor, and set f(s)= 0 
for . all other -states . Set = O for l_<i£A-M. 

Step 1 : For t=l,.2,,.., M, make the replacements 

f (s) -f- ^ P (r^ ! b (s ' , s) ) f (s ' ) / for all s. 

s ’ e p (s) 

■Step 3 : For i= m+1 , m+2 , ... A make the replacements 

h(s) -f- i P (r^^^ b(s’ ,s) ) h(s'), for all s, 
s’ep(s) 

A 

Step 4 ; If t_<A, go to step 5; otherwise set if 

I f (s)/H^__j^ ^ j 
s 

A 

and set a. . = 1 otheirwise. 
t-A 

Step 5 1 Increase t by 1, make the replacements 

H, for i = m+1, ..., A-1; 

A-x+1 A-i , , » 

and set 

Hi = I g(s)^ 



step 6 ; 
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Make the replacement 

f (s) '*■ I ■ i P (r b (s* ,s) f (s*) for all s • 
s ’ Ep (s) 

and. then return to step 2 


(Note: For simplicity we have omitted the obvious "end game" 

modifications needed when t>L which, of course, are necessary 
only if a finite frame length is used. It should also be pointed 
out that our "trick" of storing the A-M past values of the g(s} 
summation actually results in a "true" decoding delay of 2 A-M 
branches since we require the use of step 4 at 

time t - (in the algorithm) when is decoded. To reduce 

the tme decoding delay to A branches requires the storage of 
A-M+2. branches rather than 2 real numbers -per state since f(s) 
must be updated by A-M+1 branches and the A-M previous values 
of f(s) stored for each state, or, alternatively, the storage 
of 3 real numbers and considerable extra computation within 
the algorithm) . 

The algorithm as given above is directly suited for software 
implementation. It should be noted that steps 3,4, and 6 call - 
for both addition and multiplication of the computed probabilities 
that floating point arithmetic would normally be chosen for the 
calculation. (This contrasts with the Viterbi algorithm when 
fixed arithmetic is normally used as will be seen in section 3.4) 
For each t, a total of A-M+1 calculations are made in steps 3 and 
6, each involving a sum over all 2 states (whereas, as shall see, 
only one similar calculation is needed for the Viterbi algorithm. 
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A hardware realization of the real-time minimal-bit-error 

M 

algorithm could be made employing 2- micro-processors each of 
which corresponds to an encoder state s. Each micro-processor, 
would receive f(s') and h(s’) from the two micro-processors 
corresponding to the two states s' which are predecessors of s 
and with the aid of the received branches as inputs, would com- 
pute new values of f(s) and h(s) pass these values on in-turn to 
the two microprocessors corresponding to the two states for which 
s is a predecessor. Each micro-processor would execute A-M+1 
computational cycles for each t (as contrasted to one cycle for 
the microprocessors in a hardware Viterbi decoder) . 

The implementation requires the micro-processor to perform 
both multiplication and addition in floating point arithmetic. 

Similar to what is done in the Viterbi decoder, we would still 
quantize the logarithm of the probabilities into integers. The 
multiplication of the probabilities .then is performed by adding 
their quantized logarithms, whereas addition in probabilities is 
carried out by table look-up. Since the result is relatively in- 
sensitive to the addition operation and to the quantization, very 
little inaccuracy is caused by this technique. With the aid of 
Read-Only-Meraory (ROM) to store, the addition table, this implemen- 
tation of the RTMbEP algorithm, though considerable more complicated 
than the Viterbi algorithm, might not be excessively expensive. 

If the symbol size, m, considered is smaller or equal to 
the size of the memory of the convolutional code, but larger than 
one bit, this algorithm can be modified easily to minimize the symbol- 
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error-rate. In this case’ the decision rule is to choose A, 
as the estimate if it maximizes the probability 


T 


■ P(A, 


[1, t+A]^ 


~[l,t+A] ^ 


for m < A 


But since there are always, 2‘, possible in each cycle of the 
decoding procedure, we obtain very little advantage from calculating 
the last possible t+A ] ^ substracting all other 

^ [l,t+A] ^ ® already calculated from P (£j-]_ t+A ] ^ 

the RTMbEP algorithm. Thus, we shall not store H’s. But we re- 
quired the storage of 2^ real numbers to keep. P (A^,r t+A|^ which 

we shall denote as H’ (A^) . The modified algorithm is the- following: 
The RTMBBP Decoding Algorithm (forward) 


' Step 0 : Set t=l, set f(s) = ^ P (r 


b(0,s)) for the two states 


having ^ as a predecessor, and set f (s) = 0 for all 
other states. 

Step 1 ; For t=l,2,..., M make the replacements 

f(s) f(s’)r for all s. 

s ' ep (s) 

Step 2 t For each of all 2™ possible A^, set h(s) = f,(s) 
if the last m' digit of s are A^ and set h(s)=0 
otherwise. 

Step 3 : For i=M+l , M+2 , . . - , A , make the replacements 

g{s)^^ ^ ^£t+i I — h(s') for all s. 

' S’8P(S) 


Step 4 ; Set H* (A^) = I h(s), 

s 

If all the 2^ possible H* (A,j,) have not been cal- 
culated go to step 2. Otherwise, choose Ay=A^ 
such that H' (A^) is the maximum. Set i=;0. 
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Step 5 ; Increase t by 1 and make the replacement 

f{s) -(r J P (r lb (s ' ,s) ) f(s’), for all s. 

s‘ep(s) • ' 

Increase i by 1. 

Step 6 1 . If i=m go. to step 2, otherwise return to step 5. 

We should note here that although we have only 1/m 
as may decisions to make in each frame as in the RTMbEP decoding case, 
each decision requires the calculation of P(^, r^^ ^^^j}£or 2^^ 
values, of whereas only one calculation is required in the bit 
case. Thus, the number of calculations can be prohibitively large 
for even moderate byte size. Fortianately, we find that we can 
use a backwards recursion (which will be discussed in the next 
section) , instead of the forward recursion described above, to handle 
the decoding procedure more efficiently for all convolutional 

code with k^/ 1, and most efficiently for the unit-memory codes. 

But this backwards recursion offers no significant advantage for 
decoding i) convolutional codes, and moreover, since the 

forward recursion discussed in this section seems to be a natural 
approach to set up ’’real-time" decoders, we feel it warrants the 
description given here. 
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3.3 REAL-TIME MINIMAL-BYTE-ERROR PROBABILITY DECODING ALGORITHM 

FOR UNIT-MEMORY CONVOLUTIONAL CODES. 

In the previous chapter, we have noted that the unit- 
memory convolutional codes always have the largest 

minimal free distance among all convolutional codes of the same 
state-complexity, and that the state trellis structure of the 
unit-memory codes are quite different from that of (n^,l) con- 
volutional codes. Therefore, we pay particular attention to 
implementing decoding procedures for unit-memory codes. As 
commented in the last section, we are able to make use of the 
fact that every one of the 2 o states of the unit-memory code 
can reach any particular state at the next clock instant in or- 
der to derive a' backwards recursive decoding algorithm, which 
is much more efficient than the forward recursion algorithm 
discussed previously. For . the. backwards algorithm, the. joint . 
probabilities , 

^^-t' -[l,t+A]^ 

}c • 

for all the 2 o- possible can be calculated in a single 

cycle of the algorithm. 

The decision rule for this algorithm is to choose the es- 
timate a. for the information branch 

~t ^^tl' ^t2' ■^^tk ^ 

o 

as the symbol which maximizes the conditional probability 



34 



The last teinn of equation (3.19.) can be expressed as the summation 
over all states of the joint probabilities including the state to 
give 

^ ^-t'-[l,t+A] ^ = P(St+i^ -[l,t]^ ^ ^ ^^-[t+l,t+A]'^t+2|®t+l^^* 

®t+2 ‘ 
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But 


^ [ t+1 , t+ A ] ^ ®t+i+l 




= I P(£t+i'^[t+m,t+A]^ ^t+i-fl^ ®t+i+2|^t+i^ 

®t+i+2 ' 


(3.20) 


t+i^ ®t+i+l^ ^ ^ ^^ILr-f-+n 


*t+i+2 


[t+i+l,t+A]' ®t+i+2 ®t+i+l^^ 


where here again we have made use of the fact that the channel 
is memoryless and of the state property. Then it, is clear that 

J 

if we are given the transition probabilities 


P (r 


t+i ’ t+i+1 


®t+i^ 


1 < i < A ■ (3.21) 


for all possible encoded branches, we can obtain the last term 
of equation (3.19) by successively applying the recursion of 
equation (3.20) for i=A-l, A-2, ...,,1. The first term on the 
righthand side of equation (3.19) can be obtained recursively 

by 


^ ^®t+l'-[l„t] ^ I ^^®t'-[l,t-l] ^ ^^-t'^t+1 




(3.22) 


as derived in section 3.1. Hence, we now have obtained a complete 

recursive procedure for RTMBEP deeding algorithm for the unit- 

memory (n^,k^) convolutional codes. 

We now describe an algorithm for implementing the decoding 

2k 

procedure just derived,. Assuming the 2 o transition probabilities 

t 

indicated in (3.21) are available to us, we find that our algorithm 

k 

requires again the storage of two real number for each of the 2 o 
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states. We denote these two stored quantities for state s as f (s) 
and h(s). Similarly, at time t in the algorithm, the first of 
these quantities "will store the value 

f (s) 

and the other will store the value 




his) = s ) 

* |< ®t+j+l ” ®' 


t+3 


= s) 


The only additional storage required is that for the received 
branches , r ^^2 ^t+A * 


The recursions of (3'. 21) and (3.22) directly correspond 
to the following: 

The RTMBEP Decoding Algorithm for Unit-Memory Convolutional 
Codes (backwards) 


Step O i 


Step 1 . 


Wait until r^, 
and set f(s) = 

Set h(s) = I 

s ' 



P Cr 
^-t+A 


s 


A+1 received, 

0) for all states s 


' -t+A+1 ® 



set t=l. 


for all states s; 


Step 2. 


For j = A-1 / A-2 , . . . 



, 1 make the replacements 
s) h(s'), for all states s. 


Step 3. . P-ut out the estimate 


a^ = {s: s maximizes f(s) h(s))} 

and the reliability indicator 
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-£l/t+A] ^ 


Step 4 . Make the replacement 


f(s=a^) h(s=a^) 

I f(s) h(s) 
s 


f(s) ^ 


I 

s’ 




s’) 


f (s’) 


and then return to step 1. 

The obvious "end game" modification is again omitted 

here. 


It should also be pointed out that this algorithm is computationally 

similiar to the RTMbEP decoding algorithm proposed in section 3.1 

in the sense that each decision requires only one cycle of the 

algorithm, but this algorithm decodes the whole byte each cycle 

instead of one bit each time. But the trellis is fully-connected ■ 

and the summation is therefore taken over all states instead of two 

specific predecessors. The backwards recursion of step 2 enables 

us to calculate P(a. Ir,-. ji.. .i) for. all a, at the same time and, 

t I L ^ f u A J 

therefore, is more powerful than the forward recursion of step 3 
in the RTMBEP (forward) decoding algorithm. However, if we wish 
to minmize bit-error rate, the advantage of the backwards recursion 
no longer exists. Further, the algorithm shares much the same 
difficulty in hardware implementation as the forward RTMBEP 
decoding algorithm described in section 3.2, therefore, neither 
RTMBEP decoding algorithms are attractive alternatives to Viterbi 
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decoding when only hard-decisions are required. But the decoding 
delay required for unit-memory convolutional codes is usuallly 
much smaller (as counted in branches) than that required for 
(n^, 1) convolutional codes of moderate momory size so that 
the disadvantage of computing A-M branchs of state probabilities 
for each of the RTMBEP decisions is relatively minor in the case 
of a unit-memory code. 

Although we have repeatedly emphasized that the RTMBEP 
decoding algorithm requires the calculation of A-M branches 
of state probabilities, which seems a significant disadvantage 
of the RTMBEP decoding algorithm as an alternative to Viterbi 
decoding algorithm, this is primarily due to the real-time 
constraint of "fixed" decoding delay and the fact that the al- 
gorithms. are written in such a way as to minimize the use of 
storage. If we relax this constraint of "fixed" decoding delay 
but still perform "real-time" decoding, it is then possible 
to reduce the amount of computation by increasing the storage 
requirements. This is particularly true for the unit-memory 
convolutional codes because of the short decoding delay they 
require. In the following we shall demonstrate this fact. 


We note that 




-El,t+A]^ ^^^t 




-[t+l,t+A]^ ' 


and 


^^-t+1 |-[l,t+A3^ ^^-t+1 


-[l,t+l]^ ^^~t+l 


£[t+2,t+A] ^ 
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but P(A^|r, 


) and P (A. 


) are calculated 


- 'ti;t|:t[t+l,t+A] ' " ^~t+l|-[t+2,t+A]' ° — 

by the same backwards recursion o£ step 2 in the same cycle. 

It is thus possible to store the values of P — [t+2 t+A] ^ 

while calculating P (a^ t+A]^* obtain P tj-^ t+A]^ 

without going through the backwards recursion again. As will 

be seen from Table 3.2, any decoding delay greater than 8 

branches will perform virtually the same as A=8. We can then 

let A greater than 8, (which is now the maKimal decoding delay) 


and store the values of P(a.^> T,r,.^. ,^.i) for i= 2 , 3, 

t *r X ^ X L X $ t-T A J 

A-7, while calculating — ft+1 t+A]^'* These values will 

be used immediately to estimate 1+2' * * * ' ~t+A-8 Therefore, 

A-7 branches of decisions can be made with A branches of back- 
wards recursion. A can be made as large as the size of storage 
permits, in the limit', when A equals to the frame length, the 
real-time algorithm becomes the algorithm proposed by Bahl ^ al . 
[21], and the total amount, of computation is about twice that 
of the Viterbi decoder. For an example, we let A=16, and let 
the actual decoding delay vary between 8 and 16, for every 8 
decoded branches, only 16 passes of the backwards recusion are 
required in contrast to the 64 passes required in the "fixed" 
decoding delay technique. 

Natural extensions to implement similar decoders for other 
(n^,k^) convolutional codes (Mj^l) are obvious and are omitted. 
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3.4 REAL-TIME VITERBI DECODING 

As mentioned in the beginning of this chapter, the Viterbi 
decoding algorithm chooses a.j-^ information sequence 

which maximizes the conditional probability L+M] ^ ' 

The following decoding rule, v^hich we call real-time Viterbi de- 
coding (RTV) , is the natural modification of this rule to satisfy 
the "real-time constraint" that A^ be decoded from r^^ t+A] * 
Choose ^ as the byte A^ in the information segment t+A] 

which maximizes the conditional probability t+A]'^— [1, t+A] ^ * 

In keeping with our previous notation, we let 


-[t,f] ~ ^®t'®t+l'***'®t'^ 

denote the sequence of encoder states from time t to time t' 
inclusive, and let the (n ,k =1) convolutional codes be first 
considered. It follows from (3.1) that t+A+1] —(I, t+A] 


uniquely determine one another and, moreover, that ^ is the 
first m digits of ®t+2' ®t’+l^* we. inay 

phrase the real-time Viterbi decoding rules as: Choose A as 

T 

the first m digits of the state subsequence t'+l] 

state sequence t+A+1] which maximizes the conditional pro- 


bability 


^^-[1, t+A+1] -[l,t+A]^ 


^ ^-[1, t+A+1] "-[l,t+A]^ 

^^-[l,t+A]^ 


Since the denominator on the righthand side of (3.23) is in- 
dependent of _Sj^ t+A+1]' equivalently maximize the 
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o 

numerator alone. ^ 

To obtain a recursion for the numerator in (3.23) we use 

I 

the same arguments as in Section 3.2 which, for t ^ 2 give 




-n,t]'^[l,t-l]^ 




P(s^+l.£t 




[2 ^ ^ ^^®[l,t] '-[l,t-l] ^ 




if 

otherwise (3.24) 


Equation (3.24) is our desired recursion. The initial condition 
to be used for t=2 is 


■ ^^§-[ 1 , 2 ] '-[ 1 , 1 ]^ 


0 


if £eP(S2) 
otherwise 


* 


Just as for the ordinary Viterbi algorithm, the key to the 
efficient implementation of real-time Viterbi decoding is .the fact 
[readily seen from (3.24)] that the best state sequence [in the 
sense of maximizing the joint probability on the lefthand 'side 
of (3. 24 . ] t+1] ®t+l ~ ^ must be the extension of the 

‘best sequence Sj.^ with s^ = s' or s^ = s" where s' and s" 
are the predecessors of s. Hence at each time t the only storage 
required for each state is the best sequence to that state. 
Actually, since real-time Viterbi decoding requires only knowledge 
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of the first digit of the state A states previously along the 
sequencer,* we need store only A- bits per state together with 
the joint probability needed for the recursion in (3.24) . 

We write l_^ij^A and h(s) for these stored quantities 

which at time t (in the algorithm) have the values 

h(s) = P(s|jL^t+l]' ~[l,t]^ 


where s* 


[l,t+l] 


is the best path s 


~[l,t+l] 


with s^^^=s and 


B^(s) is the first digit of state ii^ the patch t+1] 

Then we may state the following: 

The Real-Time Viterbi (RTV) Decoding Algorithm 


Step It 


Set t=l, set h(s)= ^ P 


b(0,s)) for the two states s 


having ^ as a predecessor, and set h(s) = 0 otherwise. 
Set Bj^(s) = 0 for l^i_f_A and all states s. 

If t_<A, go to Step 2. Otherwise set 

— t-A ^ ®A-m +1^®^ ®A-m +2 * ’ *®A 


where s is the state for which h(s) is maximum. Set t’=0, 


Step 2 : Increase t by 1. Make the replacement 

for i = 1, 2, ...,A-1 and for all s. Increase t’ by 1. 

1 

Step 3 : For each s, make the replacement h(s)-f- 2 P (r jb (s ’ , s) ) h (s ' ) 

where s' is the predecessor of s which maximizes the 
replacing quanity, and set 

B^(s) equal to the first digit of s'. 
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Step 4: If t' = m , go to Step 1; otherwise, return to Step 2. 

The algorithm just given is directly suited for software im- 
plementation. Since the algorithm calls for only multiplication 
of the computed probabilities and selection of a maximum, loga- 
rithms may be used with the result that only computer additions 
are required and hence fixed-point arithmetic would normally be 
chosen, for the calculation. For each t, only one maximum over all, 
states need be taken, an operation equivalent in complexity to 
a sum over all states as is required A+i times in the algorithm 
of the proceeding section. In a hardware realization of the 
real-time Viterbi algorithm, the microprocessor corresponding 
to state s would receive h(s') from the two microprocessor cor- 
responding to the tv/o predecessors s' of s and, with the’ aid of 
the received branch, would compute the new value of h(s) and 
pass this value on in turn to the .two microprocessors for the 
states having, s as a predecessor. Each microprocessor would 
execute only one computational cycle ’ for each m time units and 
would be somewhat simpler than the microprocessor described in 
the preceeding section since only one quanity, h, (rather than 
two, f and h) would be processed and only additions need be 
performed. 

It should be emphasized that real-time Viterbi decoding may 

be used for L=“, i.e, when the convolutional encoder is not per- 

( 

iodically re synchronized. Moreover, ordinary Viterbi de’coding 

can ’be considered the special case of real-time Viterbi decoding for 



44 


finite L v/hen A=L+M-1. Generalization of the algorithm for 
general convolutional codes 1) is straight- forward 

and is omitted in the discussion. 

3.5 SIMULATION RESULTS 

To evaluate the performance of real-time minimum-byte-error 

probability decoding (hereafter called RTMBEP decoding) , a rate 

1/2, (2.1) convolutional coding system v/as implemented for a 

simulated additive white Gaussian noise (AWGN) channel with binary 

antipodal signaling and with 8-level output quantization for the 

bit-by-bit decoding [ (RTI^bEP) . Also implemented was an (18.6) 

unit-memory convolutional coding system] {byte size equal to 6) 

for the same channel. The results of the simulation for the bit 

case are given in Table 3.1 where Ej^ is the energy per information 

bit, Nq is the one-sided noise power spectral density, and K=M+1 

is the constraint lengh measured in information' bits. The results 

for the unit-memory convolutional code are given in Table 3.2. 

From the tables, we observe very little improvement of P„„ or 

Pj^« when the RTMBEP or the RTf'dbEP algorithm is used in place of 

Viterbi decoding. Approximately 0.1 dB to Q.2 dB of improvement 
in is observed in the bit case when the channel is very noisy 

( “ i) V7hile no improvement is observed when the signal is 

strong. Similar but slightly smaller gains are seen in the case 

of RTMBEP decoding of a unit-memory convolutional code. Since 

I 

it is of no practical interest to operate a coding system in such 
a noisy channel v/hcre the RTMBEP (or RTMbEP) algorithm shov 7 S a 
slight advantage over the Viterbi decoding, we conclude that the 
slightly better performance of the RTMBEP (or RTMbEP) decoding al- 

f * 

gorithm does npt justify the increased decoder complexity required 
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L (bits) 

No. Frames 
Decoded 

A 

•RTMbEP 

Decoding 

^bE 

Real-Time Ordinary 
Viterbi Viterbi 

OdB 3 

2400 

1 

9 

.109 

.113 .119 

2dB 3 

2400 

2 . 

9 

. 0165 

.0186 .0173 

4dB 3 

2400 

5 

9 

.00083 

.00083 .00083 

OdB 5 

2400 

1 * 

19 

.154 

. 169 

2dB 5 

2400 

2 

19 

.0165 

.0184 

4dB 5 

2400 

5 

19 

.00033 

.0033 

Table 3,1 : 

Results of Decoding of {2.1)- 

convolutional codes for 


a simulated 

AWGN channel 
No. Frames 

• 

P 

REMBEP 

BE 

Real-Time 

N 

O 

L*(bytes) 

Decoded 

A 

Decoding 

Viterbi 

l.OOdB 

400 

10 

8 

0.02950 

0.03050 

1.25dB 

400 

10 

8 

0.01925 

0,02000 

l.SOdB 

' 400 

10 

8 

0,01150 

0.01175 

1.75dB 

400 

10 

8 

0.00625 

0.00650 ■ 

Table 3.2: 

Results of 

Decoding of 

the 

(18b 6) unit- 

e 

•memory convolu- 


$ 

tional code for a simulated AWGN channel. 
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when only hard-decisions are made. 

In Tables 3,3 and 3.4, v/e show the effect of the decoding 
delay on and P^g for the M=2, (2.1) code and the (18.6) unit- 
memory code. From this table we see that the error proabilities 
decrease as longer decoding delay is employed. However, they 
saturate rapidly. From the tables, we conclude that a decoding 
delay (in branches) of about 3(M+1) to 4 (M+1) is sufficient 
for near-optimal performance with both the RTMBEP and the RTMbEP 
algorithm. 
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N 

o 

L (bits) 

No, of frames 
Decoded 

A 

(bits) 

OdB 

2400 

1 

4 

0.133 

OdB 

2400 

1 

5 

0.125 

OdB 

2400 

1 

7 

0.113 . 

OdB 

2400 

1 

9 

0.109 

OdB 

2400 

1 

19 

0.106 

OdB 

2400 

1 

29 

0.102 

2dB 

2400 

2 

4 

0.0325 

2dB 

2400 

2 

9 

0.0165 

2dB 

2400 

2 

19 

0.0140 

Table 3 . 
\ 

3: Effe^ct of Decoding Delay : 

for RTMbEP Decoding of the 

^b 

^o 

M=2, 

AWGN 

L^(bytes) 

(2.1) Convolutional Code for a Simulated 
Channel 

No. of frames 

decoded 

A (bytes) Pgg 

1.25dB 

400 

10 

4 

0.02850 

1.25dB 

400 

10 

6 

0.02475 

1.25dB 

/ 

400 

10 

8 

0.01925 

1.25dB 

400 

10 

16 

0.01925 


Table 3,4: Effect of Decoding Delay for RTMBEP Decoding, of the 

(18.6) Unit-Memory Convolutional Code for a Simulated 


AWGN Channel 
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REMARKS 


We have given a fairly comprehensive treatment of 
optimal real-time decoding of convolutional codes and introduced 
algorithms to minimize the decoding byte (as well as bit) error 
probability. We also stated a real-time Viterbi decoding algorithm 
which/ although not previously given in the literature, is pro- 
bably the form of the Viterbi algorithm which has actually been 
used in many previous investigations. 

Our conclusion from simulations of the RTMBEP decoding 
algorithm is that, although it does not reduce sufficiently 

to be a practical alternative to Viterbi decoding ,in hard-decision 
applications, the fact that it provides a direct measure of the 
quality of its decoded decisions, makes it an attractive candidate 
for the inner decoder in concatenated coding systems, as we shall 
see in the later chapters. In particular, in a system which will 
be discussed in section 5.2, the- RTMBEP decoder receiving feedback 
from an outer decoder performs much better than the Viterbi decoder 
even in terms of hard-decisions. 
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CHAPTER IV 

OPTIMAL SOFT-DECISION DEMODOLATION * 

In chapter III, we derived algorithms which minimize 
byte-error— probability and furnish the a posteriori probability 
P |— [1, t+A] ^ reliability function of the estimated 

symbol A^. To jnaice effective use of this reliability information 
for the outer decoder in a concatenated coding system, it is 
necessary to process the information in such a way that the 
required complexity of the outer decoder is within practical 
limits. In both the generalized minimum distance (GMD) and the 
errors ~and-erasures decoding proposed by Forney [2], the re- 
liability information is either hard-limited or quantized ac- 
cording to certain empirical rules. In practice, it is necessary . 
to quantize the reliability information into a few levels ac- 
cording ro a set of thresholds. Before studying the feasibility 
of optimizing the thresholds analytically, we are motivated to 
study in this chapter a similar but much simpler- problem, that is, 
.the problem of optimizing soft-decision demodulation. 

The block diagram of a one-way, coded communication 
system is given in Figure 4.1. Comparing Figure 4.1 to Figure 1.1, 
which sho>7S a block diagram representation of a one-way, con- 
catenated coding system, we find that -the two problems are si- 
milar in the sense that both systems contain a discrete channel 
which emits soft-decision symbols that are fed into a decoder. 

*Most of -this chapter is taken from [291. A portion of this 
chapter V7as presented orally by the author at the IEEE Inter- 
national Symposium on Information Theory, Notre Dame, Indiana, 
October 27-31, 1974. 
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Figureij,.!, A one-way, coded, digital conuaunications system. 
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But in the case of soft-decision demodulation , the messages are 
assumed to be sent independently, which in turn implies that the 
signals are received independently provided that the channel is 
nemoryless; whereas the messages are not independent in the case 
of soft-decision decoding . Although this subject deviates some- 
i^hat from our main-line of research, we find that it is of practical 
interest by itself and also gives guidance to the task of 
optimizing soft-decision decoding. 

From Figure 4.1, it is apparent that modulation and 
coding are both aspects of the "signal design" problem, whereas 

. t 

demodulation and decoding are both aspects of the "signal de- 
tection" problem. The natural question then is how to coordinate 
the design of the modulation system and the coding system so as 
to produce an efficient and effective communications system. 

Suppose that the modulator is M-ary; then, without loss 
of, generality, we may consider the modulator input alphabet to 
be the set {0, 1,2,,.., M-1}. Suppose the demodulator is 
restricted to J different ’ decisions, then we may take its output 
alphabet to the {0, 1, 2, ..., J— 1}. We say that the demodulator 
makes "hard— decisions" or "soft-decisions” according as to whether 
J=M or J>M respectively. Clearly, the "classical" modulation 
system design criterion of "error probability" is applicable only 
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for hard”decisions . Unfortunately for classicists, the use 

of a hard-decision demodulator generally reduces substanti tally 

the effectiveness of the coding system. 

Wozencraft and Kennedy [10] were the first to suggest that 

the proper modulation system design criterion is the "cut-off 

/ 

rate," R^, of the M- input, J-output discrete memoryless channel 
(DMC) presented by the modulation system to the coding system. 

This DMC is completely described by the transition probabilities, 
P{j[m), that the demodulator decision is j given that the mo- 
dulator input was m, 0^j<J, 0<m<M. Mathematically, the cut-off 
rate is given by 

rrain 

“ -^°^2 Q 

where Q is a probability distribution for the channel input letters, 
i.e., Q(m)^ 0 for all m and Q(0) +Q(1) + . .. + Q(M-l) = 1 

Wozencraft and Kennedy were led to the choice of (or 
as it was then usually denoted, "R ") because R is the 

upper limit of code rates for which the average decoding computation 
per digit is finite when sequential decoding is used. More recently, 
Massey [11] has pointed out -a more persuasive reason for choosing 
R^ as a design criterion. Viterbi [20] has shown that, when 
convolutional coding is used with maximum likelihood decoding 
on the DMC, then the decoding error probability is upper bounded 

by 

-NR 

where N is the code constraint length, R is the code rate (number 
of data bits per decoded letter) , L is the number of bits encoded 


J-1 M-1 


I t I /^{jlm) ] } , 

j=0 m=0 ' 
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and is an unimportant constant independent of N and L. 

Hence, as Massey observed, the single number determines both 
a range of rates over which reliable operation is possible as 
well as a measure of the necessary coding complexity to obtain a 
specified error probability. is thus even more informative than 

the channel capacity of the DMC which, although it determines the 
entire region of rates over which reliable communications is 
possible, says nothing about the coding complexity needed for 
a specified decoding error probability at any given code rate. 

In the same paper [11] , Massey established a number of 
fundamental results about modulation systems under the cri- 
terion. He gave a general expression for for unguantized 
demodulation (J=“) , and proved that, for any given M, the M-ary 
simplex .signal maximizes the unquantized for the additive white 
Gaussian noise (AWGN) channel. For binary modulation (I^=-2) and- 
any given J, Massey also gave a necessary condition for the 
demodulator decision regions to be optimal, and showed how to use 
this condition as the basis of an iterative computational technique 
for finding the optimal decision regions. 

In this chapter, we extend Massey’s necessary condition for 
optimal demodulation to the non-binary (M>2) case, and we give 
an example which shows that the condition is not sufficient even 
in the binary case. We show that, in likelihood space, the optimal 
demodulator decision regions are always bounded by hyperplanes, and 
we give some examples to illustrate the nature of these regions. 
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4.1. THE SYMMETRIC CUT-OFF RATE 


We define the symmetric cut-off rate , to be the value 
of the righthand side of (4.1) when Q is the uniform distribution 
[Q (m) = 1/M for 0_<m<M] rather than the minimizing distribution. 
Thus , 


R. 


J-1 M-1 

= log2M-log2 I C I 

^ ^ j=0 m=0 


/P(j 


(4.3) 


Evidently, Moreover, in the binary case (M=2) 

for which the uniform distribution is always the minimizing dis- 

tribution, and also R^ = R m most other cases of practical 
' o o 

interest where the modulation signal set and the demodulator 
decision regions are reasonably "symmetric". Furthermore, the 
bound of (4.1) becomes 


P 

e 


<Cj^L2 o. 


if R<Rq/ 


(4.4) 


when the code is such that each letter in the code alphabet 
appears in the same fraction of codewords, a situation that 
always occurs in the conventional convolutional codes that would 
be used in practice. Thus, both to reflect this practical 
situation and to obviate the awkward minimization over Q in 
(4.1) we henceforth take of the resultant DMC as the measure 
of quality for the modulation system. 

4.2 A NECESSARY CONDITION FOR 0PTB4AL DEMODULATION 

Henceforth, we asstame that we have made the standard trans- 
formation [26] from waveforms to signal space so that s (t) and the 
"relevant" component of r (t) in Figure 1 may be replaced by the 
corresponding vectors s and r in n-dimensional Euclidean space. 
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We let denote the transmitted signal when the modulation 
input is m. Any, demodulator then may be viewed as a partition 

m-space, where the "decision region'-' 


is the set of all received vectors r that cause the demodulator 
to emit the decision j . We now derive a necessary condition 
for the decision regions to be optimal for a given signal set and 
channel. 


Let p(r|m) be the probability density function, which 
we assume to be everywhere continuous, for the received vector 
r given that signal s„ is transmitted over the channel. The 
transition probabilities of the resultant DMC, for a given de- 
modulator, are then given by 


P ( j jm) 



P{r|m) dr. 


(4.5) 


Let a and b, a ^ b, be two output letters of the demodulator 
such that and are adjacent regions, i.e. the boundary 
between 5 ^ and is hypersurface in n-space, and let p.be any 
point on this boundary. Next, consider transferring from 

to a small region, which includes the point p_. [We 

show this situation in Figure 4.2 for the case n=2] . The resulting 


variation in the transition probabilities is then seen from 


(4.5) to be 
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6P 


, r + P(p 

(j |m) = ] - p (p 
^ 0 


n) 5V 
m) 5V 


j=b 

j=a (4.6) 

othervise 


where we have nov; assumed that each P(rjm), 0£m<ll, is continuous 
at r=p / and where 


5V = / 

ab 


is the volume of the small region If the decision regions 

are optimal, the resulting variation, of the symmetric cut 
off rate, must be O. As we see from (4.3) the condition 

= 0 is equivalent to the condition 5S=0. where 


J-1 M-1 

^ = I [ I 

j—0 .itt-0 


/P (j |hi) i 


(4.7) 


We then begin with 


J-1 M-1 

5S = I I 


6S 


j=0 m=0 


SP(j 


m) 


which, with the aid of (4.6) becomes 


Kd-l 


6S = ^ [ 


6S 


m=0 


^P (b jm) 


6S 


6F (aim) 


-]p{p|m)5V. (4.8) 


We next note that direct differentiation in (4.7) gives 


SS 


6P ( j jm) 


M~1 • 

= I I /P (j li) ] 
i=0 


(j |m) 


(4.9) 



57 


provided that P Cj |m) ^ 0. Then, by using (4.9) in (4.8) we 
obtain 


«5S 


M-1 

I I 


m=0 


1 

/P (b I m) 


M-1 

I /PCbji) 
i=0 


M-1 

1 Z/P(sii) ]• 

y'P (a|m) 


•P (p !m) 6V 

Thus, the condition that 6S = 0 for an arbitrary 6V becomes 
M-1 , M-1 _____ 1 M-1 _____ 

I [ - — - _ .1 v'PiFjT)” - — I /P (a j iT] p(p_jm) = 0, 

m=0 /^{b i m) i=0 /P (a |m) i=0 

(4.i0) 

We have thus proved: 

Theorem: The demodulator decision regions • r in 

signal space can maximize only if, for every a and b, a b, 
such that P(bjm) 0 and P(a[m) ^ 0 for 0 £ m <M, and such that 
and share a hypersurface boundary, it is the case that (10) 
holds at every point r = p on this boundary which is a point of 
continuity of P(rjm) for 0 <_ m <M. 

In the next section we shall give a more illuminating form 
of condition (4.10). 

4.3 DECISION REGIONS IN LIKELIHOOD SPACE 

For the recieved vector r, we define the waveform channel 
likelihood-ratio vector , _A (r)= (r) , A 2 (r) , . . . , (r) ] , by 
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A (r) 


p (r 

!1) 

P(r 

2) 

p (r 

0) ' 

P(£l 

0) 


p (r jM"l ) , 
p{r I 0) ^ 


(4.11) 


We note that, as pointed out by Massey [11] , the demodulator 
can always, at its "front end", map r to A(r) with no loss of 
optimality. Thus, it becomes of interest to determine the form 
of the decision regions Dq, ...., iri likelihood space 

which correspond to the optimal decision regions 3^^, ...., 

o. 

“^M-1 signal space. But, seeing from (4.11) that (4.10) may, 
after division by p(p,l0) (which we. now assume to be non~zero) be 
rewritten as the linear equation 


M-1 , 

I r — ^ 

m=l /P(bjm) 


M-1 

I /P (b I i) 
i-0 


/P (a I m) i=0 


M-1 

I /P(aliyj A^(p) + 


/P (b i 0) i-0 


M-1 , 

I /FiblTT 


/P (a 0) i=0 


M-1 ___ 

I /Pia^i) = 0, 


(4,12) 


we have immediately our main result 

Corollary 1: The demodulator decision regions 

in signal space can maximize only if, for every a and b,a 7 ^b, 
such that P(blm) 0 and P (a[ m) 0 for 0_<n<M and such that 

and.;,^, share a hypersurface boundary, if is the case that 

every point r = ^ on this boundary, which is a point of continuity 

of <^(£) lies on the hyperplane defined by (4.12) 

In other words, the optimal decision regions in likelihood 

space are always bounded by hyperplanes. This fact has considerable 

practical significance as it is difficult to implement circuitry 
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which determines to what decision region some vector ;A(r) 

% 

belongs except in the case when the decision regions are bounded 
by hyperplanes. 

Condition (4.12) can be placed in an even more transparent 
form. We note that when (4.12) is satisfied, then 


T 

m 


1 

/P (b| 0) 


1 

/^(b jm) 


M“1 


I 

i=0 


M-1 


I 

i=0 


/ P(b|i) 


/pTb]i) 


1 

>/p{ajo) 


1 

/P (a jm) 


M-1 

I /P(ali) 
i=0 

M-1 

^ /P(aji)" 
i=0 


(4.13) 

(provided the denominator is non-zero) is just the intercept on 
the m-th axis in likelihood space ot the boundary hyperplane. 

[See Figure 4,3 for a graphical interpretation of Thus, 

we have 

Corollary 2 ; The demodulator decision regions D^, D^,..., 

in likelihood space can maximize only if, for every a and b, 
afi^h , such that P (a jm) 0 and P(bim) 0 for 0 ^ m <M, and such 
that and share a hypersurface boundary, it is the case that 
this boundary is a hyperplane whose intercepts with the coordinate 
axes are given by equation (4.13) 
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Figure 4.3. A hyperplane boundary in likelihood space 
separating the decision regions D 
and -Dj^ for the case M=3. ^ 

Analogous to our definition of ^(r) we now define the 

likelihood vector, ^(j) - EX^(j), A 2 (j), 

of the DMC, which is created by the modulation system, 

by 


X k ) 

P( j |0) (4.14) 

where we assume P(j|0) =}= 0 for 0 j < J. From (4.12) 

(4.13) and (4.14) it follows after some tedious algebraic 

manipulation that, when the decision regions are optimal 


M~1 

I 


m=l 





1 


(4.15) 


For the case of binary modulation (iyt=2) , we note that (4.15) 
reduces to the necessary condition for optimality, 

T = /X(b) X(a) (4.16) 

that was given by Massey [11]. 
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4.4 EXAMPLES AND AN ITERATIVE OPTIMIZATION TECHNIQUE 

In this section we give some examples to illustrate the . 
use of the necessary condition for demodulator optimality given 
by Theorem 1 and its corollaries. We begin with a binary 
signaling example in which Massey's algorithm [11] based on equal- 
tion (4.16) can be used conveniently to demonstrate effects of 
quantization. In the later examples of non-binary signaling 
cases, we also formulate a systematic method for finding the 
optimal demodualtor decision regions by iteration from an initial 
guess. 

Example 4.1 ; Binary anti-podal signals in additive white 
Guassian noise (AWGN) . In this case, the signal space may be 

taken as one-dimensional. The received signal r may be written 

r = s + n 

where =-/ e , E is the. signal energy, and n is a 

zero-mean Gaussian random variable with variance N /2. N is 

o o 

the one-sided noise power spectral density. The likelihood 
ratio, A(r) = p (r }o)/p (r j 1) becomes 

. , , (4/e/N )r 

Because likelihood space is one-dimensional, the hyperplanes be- 
tween decision regions are just points or "thresholds". Because 
A (r) is monotonic in r, each such threshold T between decision 
regions in likelihood space can identified with the threshold 

(Ny 4/F) log^ (T) 
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between the corresponding decision regions in signal space. 

The demodulator can then be specified by the J-1 theresholds 
t(l), t(2),.,., t(J-l) in the manner that the demodulator 
output is j when t(j)<r £ t(j+l) with the convention that 
t(0) = and t(J)= +“ . It truns out to be more convenient 
to use the normalized thresholds t' (j)= (/2/N^) t(j)- in order 
to weaken the dependence of the optimal thresholds on the 
energy-to-noise-power- spectral-density ratio . 

In Table 4.1, we give the normalized threshold values 
t' (1) / t'(2),..., t' (j-1) that maximize for the case J=3, 
J=4 and J=8 over a wide range of ratios. These optimal 

thresholds were determined by Massey's iterative technique 
[11]. The resulting for each case is given in Table 4.2 
where we have included, the value of for an unquantized 
demodulator (J=“>) to show the loss due to quantization. For 

f\j 

comparison, we give also the value of R^, acheived by using 
the heuristically-chosen "good" thresholds given by Jacobs 
[27]. For the case J=4, Jacobs normalized thresholds are 
t* (3)=-t' (1)=1 and t' (2)=0; while for J=8 they are t' (7)=-f (1) 
= 1.5, t' (6)=-'f (2)=1, t* (5)=-f (3)=0.5 and t' (4)=0. From 
Table 4.2, we see that the optimum thresholds offered scant 
superiority in over Jacobs' thresholds. 
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Table 4.1 The normalized thresholds for J-nry demodulation, 

^ maximizing for binary antipodal signals transmitted 

through the SwGN. 
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J=2 

J=3 


0 

r=8 

1 J=<» J 

o o 
_ (DB) 

Optimal 

Optimal 

Optimal 

I Jacobs 

Optimal 

. 

j Jacobs - 

I Unquantized | 

-5.0 

0.13641 

0.17195 

0.18657 

0.18657 

0.20298 

0.20298 

0.21015 

c4.0 

0.16905 

0.21251 

0.23029 

0.23029 

0.25015 

0.25015 

j 0.25878 j 

-3.0 

0.20864 

0.26133 

0.28272 

0.28272 

0.30645 

0.30645 

0.31670 j 

-2.0 

0.25615 

0.31929 

0.34467 

0.34466 

p. 37258 

0.37258 

0.38451 j 

-1.0 

0.31240 

0.38692 

0.41643 

0.41640 

0.44853 

0.44853 

0.46208 j 

+0.0 

0.37786 

0.46396 

0.49740 

0.49730 

0.53320 

0.53320 

0.54806 

+1.0 

0.45230 

0.54899 

0.58553 

0.58533 

0.62385 

0.62385 

0,63940 

+2.0 

0.53445 

0.63894 

0.67697 

0.67659 

0.71572 

0.71572 

0.73100 

+3.0 

0.62165 

0.72888 

0.76593 

0.76528 

0.80227 

0.80227 

0.81607 1 

+4.0 

0.70961 

0.81238 

0.84547 

0.84449 

0.87633 

0.87633 

0.88748 j 

+5.0 

0.79273 

0.88287 

0.90927 j 

0.90799 

0.93234 

0.93234 

0.94109 j 


Table 4.2: Values of R for the DMC obtained by optimally quan- 

tizing the output of AWGN channel for binary anti- 
podal signals employing the thresholds listed in Table 
4.1; values of R obtained using Jacob's heuristically- 
chosen thresholds are listed for comparison. 
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Example 4.2 ; Hard-decision demodulation (i.e./ J=JMi) for M-ary 

phase modulation in additive white Gaussian noise (AWGN) . In this 

case, the signal space is 2-dimensional and the signal vector, 

s for<0<m, M, may be taken as the point on the circle of radius 
— m ~ 

/e ( where E is the signal energy) at an angle of (27r/M)m. The' 
ternary (i.e., M=3) case is shown in Figure 4. The heavy lines 
in this figure are the boundaries of the decision regions for 



F igure 4.4 : Maximum Likelihood Demodulation of ternary phase-modulated 
signals, 

a maximum likelihood (ML) -demodulator which, of course, is the 
h^td-decision demodulator that minimizes errors probability when 
the signals are equally likely. 
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We now show that the ML demodulator for phase modulation 

is also the hard-decision demodulator which maximizes R . Let 

o 

s and ^ be any two adjacent signals, i.e, their phase dif- 
— a *~t> 

ference is By the symmetry of the signal set and by the 

spherical symmetry of the additive white Gaussian noise, it 
follows that the ML demodulator causes the probabilties P(b[0)^ 
P(b[l),..., P(b|M-l) to be a permutation of P(ajO), PCajl), 
P(a)M-l), and also that for each m such that P(bjm) ^ P(a|m) 
there is a corresponding m' such that P(bjm) = P(ajm’)/ P(a|m)= 
P(blra’) and pCpjm) = p(p|m')'for p on the boundary between and 
i>& . Thus, the terras in the summation on the lefthand side of 
(4.10) either vanish' singly [when P(b|m) = P(a|m)3 or cancel in 
pairs. Thus, the ML decision regions satisfy the necessary 
condition for maximizing given by Theorem 1. Symmetry con- 
siderations indicate this is the only local maximum of and 

o 

hence is the global maximum. 

As a specific nximerical example, we take the M=3 case 
of Figure 4.4 where E/n^ = 1, being the one-sided noise power 
spectral density so that the variance of the noise in each di- 
mension of signal space is N^/2. The value of yielded by 
the optimal hard-decision demodulator is 0.3971. The unquantized 
for this case can be found from Massey’s results [11] to 
be 0.6254 so that the penalty for hard-decisions is 1.97 dB. 

Example 4.3 ; Quaternary demodulation (J=4) for ternary 
phase modulation (M=3) in AWGN. Symmetry considerations suggest 
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that the optimal decision regions will consist of three regions, 
and S' 2 r having 120 ° rotational symmetry and containing 
the signals and I’espsctively, together with an "erasure" 

region containing the origin in signal space. In this case, 
there will be probabilities p and q such that P(ajm) = q for all 
m, P(j|m) = p for j 7 ^ m and j 3, and P(jjm) = l-2p - q for j = m. 
Substituting these parameters into the necessary condition for 
optimality (4.13) we find that the resulting optimal intercepts 
correspond to the straight lines. 

+ A 2 = c 
cAf ~ ^2 ~ ^ 

~h -^c'A2 = 1 

where 

c = 2 /p/ / ~i- 2 p-q 

Thus, the optimal decision regions in likelihood space are known 
lup to the parameter -c. By trying various choices of c for the 
specific case E/N^ = 1 and calculating for the DMC resulting 
from the demodulator corresponding to these decision regions, we 
find that, for the optimal decision regions, c= .486 and the 
attained value of is 0.4402 which is a 0.45 dB improvement over 
hard decisions. 'In Figure 4.5 (a), we show the optimal decision 
regions in likelihood space, while in Figure 4.5 (b) we show the 
corresponding regions in signal space. 





slope 2.06 


Slope 0.486 



2.06 


I \ I 
Xv V 


Figure 4.5 


Optimal decision regions for quaternary 
demodulation of ternary phase modulation 
in AWGN with E/Nq== 1 shown (a) in likelxhood 
space, and (b) in signal space.- 
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We now describe a general iterative procedure that may 
be used to find the optimal decision regions for J-ary demodulator 
given a particular M-ary signal set and a given channel. The basic 

idea is quite simple.. Given decision regions ^j-1 

bounded by hyperplanes, we note that, for some a?^b such that D 
and are adjacent regions, the intercepts of the bounding 
hyperplane with the coordinate axes in likelihood space will satify 
(4.13) if the decision regions are optimal. If they are not optimal, 
we can use the numbers determined by (4.13) as the intercepts of 
a hyperplane which will be a better approximation to the optimal 
bounding hyperplane between D and D, . Our procedures may be stated 

3 , ^ 

as : 

Iterative Demodulator Optimization ; 

Step 0 : Make an initial guess, / • • • • / ' 

for the optimal hyperplane-bounded decision regions in likelihood 
space. Set k=l. 

Step 1 ; Calculate P(jjm), 0£j<J and 0^<M, for the DMC 

(k) fk) ^k) 

created by the decision regions , . . . , ; 

Step 2 : Choose an a and b, a?^b, such that and 

are adjacent and calculate from 

equation (13) . [Note: If the decision regions are optimal, 

I 

then these T'*s will be the intercepts of the boundary between 
(k) fk) 

and Dj^ with the coordinate axes.] 

Step 3 ; Take the boundary between and as 

the hyperplane whose intercepts with the coordinate axes -are 
(k+1) ,^(k+l) (k+1) 
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Step 4 ; Repeat steps 2 and 3 until all such pairs a 
and b have been considered. 

f 

Step 5 ; If are . "sufficiently close" to 

(k) fk) (k) 

Dq , stop and take the fonaer decision regions 

as the result of this optimization method. Otherwise, increase k 
by 1 and return to step 1. 

Two remarks about the above iterative method are in order. 
First, the most time-consuming part of the' procedure is the cal- 
culation of the transition probabilities P(j|m) in step 1. This 

fkl 

would ordinarily be done by mapping from the decision regions ' , 
(k) (k) 

in likelihood space to the corresponding decision 
regions , . . . , in signal space, then evalu- 

ating the intergral in (4.5) either analytically or numerically. 
Secondly, we note that when M=2, the above iterative procedure 
requires more calculation than the iterative method given by 
Massey [11] which is based on equation (4.16)^, Unfortunately, 
Massey’s method does not generalize to cases where M>2. 

We now give two examples to illustrate the use of the 
above iterative optimization method. The first of these is a 
"hard-decision" case which serves also to illustrate the fact that, 
when the signal set is not sufficiently symmetric, even in this 
case the decision regions which minimize demodulator error 
probability may not maximize 

Example 4.4 : Hard-decision demodulation for the two-dimen- 
sional signal set Sq = [0,0], s^ =[2^0] and £2 = [0,2] in 
AWGN with variance N^/2 = 1/2 in each component. [The average 



71 


signal-energy-to-noise-power-spectral-density-ratio, is 

2.67 (or 4.26 dB) . ] In this case, the received vector r = [x,y] 
has a likelihood ratio vector 

Air)= [A_j^,A2] = r e^^"^]. 


Thus, the boundary straight-line (or hyperplane in two-dimensional 
space) 





1 


corresponds to the curve 


1 ^4x-4 ^ 1 

T ^ + -ip 

1 ^2 


4y-4 _ 


= 1 


in signal space. 


(4.17) 


As the boundaries between decision regions in the initial- 
ization step 0 of the algorithm, we choose those which minimize 
demodulator error probability,' viz., the straight lines A^^= 1, 

Al -A 2 shown in Figure 4.6(a). Note that the boundary between 
and D 2 is a reflection around the line =A 2 of that between 
and D^. This symmetry is preserved at successive iteration 
of the algorithm because of the corresponding symmetry of the 
signal set about the line x=y and the symmetry of the AWGN. Thus, 
it suffices to determine the boundary between and D^. Letting 
(Tf, T 2 ) be the intercepts of this boundary with the A^'.and.A 2 
axes, respectively, and taking (1,-10 ) as an approximation to 
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the initial , we find, from application of the interative 

procedure in which we use (4.17) to determine the region over 
which the integral in (4.5). is to be evaluated numerically, the 
following succession of intercepts: 

(1, -10^) initialization 

(0.8307, -0.5732) 1st iteration 

(0.8145,-0.4030) 2nd iteration 

(0.8252, -0.3501) 3rd iteration 


(0.8646, -0.3100) 


15th iteration 


up to the point where there is no further change in the 4th signi- 
ficant digit on further iteration. In Figures- 4.6 (b) and 
4.6 (c) , we show the optimal decision regions for this example, 
as found by the iterative optimization procedure, in likelihood 
space and in signal space, respectively. 

The values of obtained as successive iterations were 

o 

as follows: 

0.6388 
0.6462 
0.6476 
0.6480 


initialization 
1st iteration 
2nd iteration 
3rd iteration 


0.6482 


15th iteration 
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r\j , 

The optimal value of is about 1.5% (or ,06 dB) above that 
obtained for the hard-decision demodulator which minimizes error 
probability. We see also that, for this example, the iterative 
procedure converged rapidly to the optimal demodulator — after 
only one iteration the resulting demodulator was effectively 
optimal . ' 

As can be seen from Figure 4.6 (c) , the optimal decision 

boundaries have asymptotes which are straight lines {hyperplanes 

in two-dimensional space) . These asymptotes are shown- by the dashed 

lines in the figure. If one uses these asymptotes as the boundaries 

of the decision regions for a sub-optimal demodulator, one finds 

the resultant R to be 0.6459 which is only .015 dB inferior to 

o 

the optimal hard-decision demodulator. We shall later discuss the 

practical significance of the near-optimality of these asymptotic. 

linear boundaries in signal space. 

Example 4.5 ; Quaternary demodulation (J=4) for the same 

ternary signal set (M=3) and noise as in example 4.3. 

In Figure 4.7 (a) , we show the J=4 decision regions used 

to initiate the iterative optimization procedure. The optimal 

decision regions in likelihood space and in signal space are shown 

in Figures 4.7 (b) and 4.7 (c) , respectively. ' Convergence to 

four significant digits of accuracy in the intercepts of the 

boundary lines with the coordinate axes required 25 iterations. 

% 

The values of R at successive steps were as follows: 

o 

0.6535 initialization 

0.6986 1st iteration 

0.7045 2nd iteration 

0.7057 3rd iteration 
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0.7062 25th iteration 

Me see that the decision regions after only two iterations were 
effectively optimal. The optimal value of for quaternary 
demodulation is about 8% (or 0.34 dB) above that for optimal 
hard-decision demodulation. 

In Figure 4.7 (c) , the dashed lines (or hyperplanes in 
two-dimensional space) again are the asymptotes to the optimal 
decision boiindaries in signal space. If these asymptotes are 
used as the actual boundaries betwen the decision regions in . 
signal space ^ we find of the resultant demodulator to be 
.7031 which is only .02 dB below optimal. 

In Examples 4.3 and 4.5 we have seen that the linear 
(hyperpla ns) asymptotes to thie* optimal decision regions in signal • 
space themselves bound the decision regions for a demodulator 
-which is virtually optimal. The practical significance of this 
fact is that the resulting sub-optimal decision rule can be as 
easily implemented directly in signal space as can the optimal 
decision rule in likelihood space. There is no need for the con- 
version from signal space to likelihood space in order to obtain 
conveniently-implemented decision regions with linear (hyperplane) 
boundaries . 

In fact, it can be shown generally, for AWGN in an n-dimensional 
signal space, that the optimal demodulation regions in signal space 
are such that each bounding hyper surf ace has (at most) 2^ ^ hy- 
perplane asymptotes. We conjecture that these hyperplane asymptotes 
form the boundaries of decision regions '■for a demodulator that is 
virtually optim’jm, and hence that the mapping from signal space 
to likelihood space is not necessary to obtain virtually optimal 



Slope 7.868 


Slope 0.1271 
1.4361 ”l 
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demodulation together with an easily-implemented decision rule. 

4.5 A COUNTERE^IPLE TO THE SUFFICIEISrCY OF OPTIMALITY CONDITION 
(4.10). 


As we have pointed out above, condition (10) is actually 
the condition for an extremum of and hence not in general 

a sufficient condition for optimal demodulation. In the examples 
which we have studied wherein the noise was additive with a "smooth" 
density function, there has generally been only one set of decision 
regions satisfying (10) so that the extremum was necessarily the 
global maximum of We now give an example to show, however, 

that it'is possible for (10) to be satisfied for decision regions 
that define only a local maximum, or even a local minimum, of 

R . 

o 


Example 4.6 : Hard-decision demodulation for binary signals 

such that the conditional density functions for the likelihood ratio 
A are 


P ■'> 


0.25 
2.75 
' 0.25 


0 < A 5 0.9 
0.9 < A _< 1.1 
1.1 < A < 2.0 


0 2.0 < A 

and p^(\) — Pq (A) when the signals and are transmitted, 
respectively. [These are valid choices for these conditional 
density functions as they satisfy the constrains 


jf p^(A )dA= jr ( /V )dA _= 1 
0 0 


A = P^(A )/Pq( A) 


and 
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that are the only ones that must be observed in the binary 
case] . 

For hard-decision demodulation with binary signalling, 
condition (4.10) reduces to Massey's condition (4.16) namely, 

T = /A (T) A (0) (4.18) 

Where T- is the threshold between decision regions in likelihood 
space. 

In Figure 4.8 (a), we show the conditional density 
functions for the likelihood ratio A, and in Figure 4.8 (b) 
we show /a ( 1) A (0) as a function of the . threshold T between 
the decision regions in likelihood space- We see that condition 
(4.18) , the necessary condition for optimal demodulation, is 
satisfied at three places, vis., 0.56, 1.06, and 1.16; the 
corresponding values of are 0.0123, 0.. 0066, and 0.0069, 
respectivley. The third of these corresponds to a local, but 
not global, maximum of The second corresponds to a local 

f\f 

minimum of R^, while the first corresponds to the desired global 
maximum of R^- That is, ‘T=0.56, is the threshold between decision 
regions in likelihood space for the optimal demodulator. 



PqCa) 


2 


1 



(b) 

Figure 4.8: The binary, hard-decision, demodulation situation 

used to demonstrate the Insufficiency of optimality 
condition (lO) . 
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4 . 6 SUMMARY 

In this chapter, we have derived a necessary condition 
for optimal J-ary demodulation of M-ary signals, where optimality 
is taken to mean maximality of the symmetric, cut-off rate, 
of the resulting discrete memory less .channel. By means of a coun- 
terexample, _ we have sho\?n that this - condition is not in general 
sufficient for optimality. We, have also used this necessary con- 
dition for optimality as the foundation for an iterative optimization 
method to find the optimal demodulator decision regions from an 
initial "good guess". 

In general, the optimal demodualtor decision regions are 
bounded by hyperplanes in likehood space. For the' important case 
of additive white Gauusian noise, the corresponding optimal decision 
regions in signal space have hyperplane asymptotes. In some examples 
we have shown that the regions in signal space bounded by these 
asymptotic hyperplane define demodulator decision regions which 
are virtually optimal, and we conjectured that this happy state of 
affairs (which permits near optimal performance with a decision 
rule that can be simply- implemented directly in signal space) holds 
in general. 

In regards to the application of this results to the problem 
of soft-decision decoding for the concatenated separate coding system, 
we note here that the super-channel of Figure 1.1, is again processed 
by a decoder? therefore, the criterion of optimality of maximizing 
of the resultant discrete channel is still valid. Unfortunately, 
since this super-channel is no longer memory less, the analysis ap- 
pears to be extremely difficult and we shall not pursue it further. 



CHAPTER V 


PERFORI'IANCE OF CONCATENATED CODING SYSTEMS 
ON A SIMULATED AWGN CHANNEL 

In this chapter, the performance of several' RS- convolu- 
tional concatenated coding systems utilizing the unit-memory codes 
found in Chapter II will be compared with similar systems utilizing 
the usual (n^/1) convolutional codes. The first of basic system 
is the hard-decision Viterbi decoded concatenated system with no 
feedback between the inner and outer decoders as proposed by 
Odenwalder [5] - In the second system studied, the 'Viterbi decoder 
is replaced by a RTMBEP decoder while the RS ”errors-only" [2] 
decoder is replaced by an "erasures-and-errors" [ 2 } decoder- In the 
third, fourth, and fifth systems, the inner decoder are restarted 
whenever the outer decoder corrects an error and feeds back the 
correction to the inner decoders. In the sixth and final system, 
we annex a tail to the unit-memory code (the code is, then, no 
longer of "true" unit memory) to enhance the capability of the 
Viterbi decoder to detect unreliable decoded symbols as was 
proposed by Zeoli [8] and Jelinek [9] for the {n^, 1) convolutional 
code. The various systems studied are summarized in Table 5.1- 
In each case, we choose the (18.6) unit-memory code as the inner 
code because it has practically minimum complexity in terms of 
decoder implementation and because of its reasonably large free 
distance . We choose the Reed-Solomon codes over 

g 

GF{2 ) with block length 63 symbols as the outer codes so that the 
symbol size of the RS code is matched to the byte-size of the 


81 



SYSTEM 

Inner 

Decoder 

Outer 

Decoder 

Feed- 

back 

Erasure 

Inner Code 
rail Annexa— ’ 
tion 

I 

Viterbi 

EO 

NO 

NO 

NO 

• II 

RTMBEP 

EE 

NO 

YES 

NO 

II 

Viterbi 

EO 

YES 

NO 

i 

IV 

RTMBEP 

EO 

YES 

NO 

NO 

V 

RTMBEP. 

. EE . 

YES 

YES 

NO 

VI 

Viterbi 

EE . 

YES 

YES 

YES 1 

* EO: Errors Only Decoders 1 

EE: Erasures and Erros Decoders | 


Table 5.1: Various Block-Convolutional Concatenated Coding 

Systems Studied in Chapter 5. 
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information sequence of the unit-memory code. We also assume 
the interleaving to be "perfect". In other words, we assume 
that the decoded symbols at the output of the scrambler are 
statistically independent. With this assumption, we can calculate 
the error probability of the Reed-Solomon block by means of 
the statistics for the inner decoder which we obtain from 
simulations. Figure 5.1 shows an encoder and a Viterbi decoder 
for the unit-memory code interleaved to degree. N. Note that the 
interleaving is done on a byte-by-byte basis, so that it "destroys 
information" to a much less extent than if it been done on a bit- 
by-bit basis. Further, we make use of the fact that almost all 
the incorrectly decoded Reed-Solomon codewords are d^^^ symbols 
away from the correct codewords to estimate the byte-error pro- 
bability of the over-all system, where d^^^ is the minimum distance 
of the RS code. (We assxime that the outer decoder always decodes, 

r 

even when two codewords are equally likely. ) 

5.1. ODENt-?ALDER' S CONCATENATED CODING SYSTERI AND SOFT-DECISION 
MODIFICATION WITH THE RTMBEP, DECODING ALGORITHM. 

The concatenated coding system . proposed by Odenwalder [5] 
is shown in Figure 1.1. His original scheme employed a hard-decision 
Viterbi decoder as the inner convolutional decoder and a t-error 
correcting "errors-only" RS decoder as the outer block decoder. 
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Then^ the probability of error of a Reed-Solomon block is 
given by 

63 63 . 

■^BERS ^ ^ i ^ P (1-p) / (5.1) 

i=t+l 


where p is the byte-error probability of the Viterbi decoder 
and t == Then, the byte-error probability of the 

concatenated coding system assuming complete maximum likelihood 
(ML) decoding is given by 


P 


BE 


2ttl 

63 


^BERS 


(5.2) 


The byte-error-probability of the inner convolutional coding 

system, at signal energy per information bit to one-sided noise- 

power- spectral-density ratios for the inner coding system, 

(Ej^/N^)j, equal to 1.0 dB, 1.25 dB, 1.50 dB, and 1.75 dB are 

given in Table 2»3f which in turn correspond to signal-energy 

per inner channel digit to one-sided noise-power-spectral-density 

ratios E /N , of -3.77 dB, -3.52 dB, -3.27 dB, and -3-02 dB. 
s o 

This simulation result is used to determine the perfomance of the 
system. For (Ej 3 /Nq)j == 1.25 dB, we show in Figure 5.2 the cal- 
culated error probability versus the over-all signal-energy per 
information bit to one-sided noise-power-spectral-density , which 
is given by 




'RS 


I 


®RS®CON 




(5.3) 
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P 


BE 



I 




i 


I 

1 

I 

i 

I 




10 


7 


Concatenated with 
4-error-correcting 
RS code 



6-error-correcting 
Rs- code 

8-error-correcting 
RS code 


Viterbi Decoding for 
(18.6) Unit-memory 
Convolutional Code 
System I with (3.1) 

code 


System I with M=7, 
(3.1) code 

System I with (18.6) 
Unit-memory code 


System II with (18,6) 
unit-memory code 


{ j , 4 , j 1 

1.9 2.0 2.1 2.2 2.3 2.4 2.5. 2.6 (E^/N ) 

' b' o o 

dB 

Figure 5.2: The Performance of Concatenated coding Systems 

I and II with different RS code for a simulated 
AWGN channel with E /M = - 3.52 dB, and the 
performance of Vitefbi^decoding for the (18.6) 
unit-memory code without concatenation. 
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where Rj^g and the rates of the RS code and the con- 

volutional code, respectively. 

Also plotted in Figure 5 . 2 are the performance- curves 
at the same (Ej^/N^) ^ for similar concatenated coding systems 
using the M=6 {3.1) convolutional code and the M=7, (3.1) 
convolutional code as inner codes. The performance of the coding 
systems employing these three convoltional codes concatenated 
with a 6 -error correcting Reed-Solomon code is shown in Figure 
5.3. From both Figures, an approximatley 0.3 dB advantage in 
(Ej^/Nq)^ for the (18.6) unit-memory code, consistent with the 
result shown in Table 2.3 is observed. But the performance of 
concatenated systems is, in general very sensitive to 
this 0.3 dB advantage corresponds to a factor of 10 to 100 
advantage in "the error rate of the outer decoder, which, of 
course, should not be neglected. Even when we compare the 
performance of the unit-memory code to that of the M=7, (3.1) 
code which is of the same free distance as the unit-memory code 
.approximately O.ldB to 0.15 dB advantage can be observed due 
to the byte-oriented structure of the unit memory code. 

We now replace the Viterbi decoder by a "Real-Time Minimal- 
Byte-Error Probability" decoder in^ the concatenated coding system. 
When the reliability function- of the decoded symbol t+A]^ 

is less than certain threshold, T, the decoder emits an erasure. 
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p 

BE I 



\ (18.6) tmit-memory code 



1.9 2.0 2.1 2.2 2.3 2.4 2.5 2.6 dB(Ej^/N^)^ 

Figure 5.3: The performance of concatenated coding 

system I empolying 3 different convolutional 

codes and a 6-error correcting RS code with 

■ " error S“only" decoder for a simulated AWGN 

channel with -3.77 dB<E /N <-3.02 dB. 

— s o— 
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An erasures-and-errors Reed-Solomon decoder which can correct 
t errors and e erasured provided 


2t+e<d 

is used as the outer decoder- Then the probability of error 
of a Reed-Solomon block, assuming complete ML decoding, is given 
by 


BERS 


63 

I 

d’=d. 


mxn 


d V2 

I 

t=0 

e=d'-2t 


63 

( ) 

t,e 


t e 

P q (l-p-q) 


63— t— e 


(5.4) 


where p is again the probability of byte— error of the inner decoder 
and q is the probability of erasure of the inner decoder, and 

N Nl • ‘ 

^ ^ ~ tie! (N-t-e) 1 

"t / G 

Here again, we assume that the decoder always decodes and in case 
of a tie it always makes an error- Then, the byte-error probability 
of the overall system is again obtained from (5.2). 

The by te-error-prob ability p and the erasure probability 
q depend on the particular threshold, T, selected. The optimal 
threshold is a function of ^nd the minimum distance, 

of the Reed-Solomon code. Roughly speaking, for a given block 
length, when gets larger, the over-all block error probability 

is minimized at a higher erasure rate. But there is no. simple way 
to determine the optimal threshold analytically. We then find the 
p and q for T = 0.5, 0.7, and 0.8 by simulation and use these values 
of p and g to calcululate the byte-error probability of the coding 


system. 
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In Table 5.2, we show the result of this calculation. 

We see that for in the range between 1.25 dB and 1.75 dB, 

T = 0.7, is the best threshold among the three candidates. The 
performance of this concatenated coding system with T=0.7 is also 
plotted in Figure 5.2. The improvement of the performance due to 
the erasure scheme, as observed from Figure 5.2, is dependent on 


the error correcting capability of the outer coding system as well 
as and is approximately 0.1 dB, This slight improvement 

is probably not significant enough to justify the increased com- 
plexity of the convolutional decoder. However, as we shall see 
in the later sections, the RTMBEP decoder coupled with a "erasures- 
and-error” block decoder performs much better than the Viterbi 


decoder when feedback from the outer decoder is available. 


(V^O^ 

I*? 

(dB) 



0,80 

x.oo 

o.?o 


0.50 


0.80 

1.25 

0.70 


0.50 


0.80 

1.50 

0.70 


0.50 


0.80 

. 1.75 

0.70 


0.50- 




Over-all Symbol -Error Probability 
d=9 d=i3 d=17 


0 . 244 < 10 *“^ 0 . 193 < 10 *^ 


r-^ 0. 334^1 0”® 0.l96no"^^ 


Table. 5.,2 :The effect of erasing thresholds T on the symbol- 

error probability of the over-all concatenated coding 
systems for various {B.j/N^)jwith various outer codes. 
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5. .2 Feedback from the outer decoder to the inner decoder 

Because of the nature of the convolutional code and 
the Viterbi decoding algorithm, once an "error event" occurs 
the decoder often makes a number of closely spaced erroneous 
estimations before it recovers to correct operation. Since 
the outer decoder of a concatenated coding system is design- 
ed in such a way that it is able to detect and correct almost 
all of the errors made by the inner decoder, it is then of 
significant advantage if the corrected estimation of- the 
outer decoder is feedback to restart the. inner decoder to 
avoid these "bursts" of errors. Figure 5.4 illustrates the 
general concept of such concatenated coding systems. 



Figure 5.4. 'A typical Concatenated Coding System with 

Feeback from Outer Decoder to Inner Decoder 
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We have implemented a software Viterbi decoder and 
a software RTMBEP decoder which can be restarted with feed- 
back. We carried out the simulations for this type conca- 
tenated coding system by assuming that the outer decoder 
always makes correct decisions. (This. is a valid assump- 
tion since the probability of byte-error for the over-all 
system is negligible compared to that of the inner convolu- 
tional decoder.) We summarize the results of this simula- 
tion for the (18,6) unit-memory convolutional code at an 
(E./N ).^ of 1.25 dB in Table 5.3. We see that the RTMBEP 
decoder performs much better, than the Viterbi decoder in 
this case. That is, the feedback from the outer decoder 
plays a much more important role in helping the RTMBEP 
decoder to recover from the error than it does for the 
Viterbi decoder . In Figure 5.5, we show the perf oimiance 
of these two systems concatenated with the " error s-only" 
block decoder. From the figure, we conclude that the 
improvement in performance due. to the feedback is appro- 
ximately 0.3 dB for the Viterbi decoder and about 0.5 dB for 
the RTMBEP decoder. Also listed in Table 5.3 is the error 
probability obtained for the M=7, (3,1) convolutional code 
(which has the same free distance as the (18,6) unit-memory 
code) with the same kind of Viterbi decoder with feedback. 
From Figure 5.5, we see this 0. l dB difference in perform- 
ance between the unit-memory code and the M=7, (3,1) code 
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Byte-Error-Probability 95% Con- Byte-Error 95% Con- 
fer (18/6) Unit-Memory fidence Prob. for fidence 
Code Interval M=7, (3,1) Interval 

Code 


Viterbi 

Decoding 


0.0110 


+0.0023 0.01325 +0.0025 


RTMBEP 

Decoding 


0.0075 


+0.0075 


Table 5.3: The Performance of Viterbi Decoder and RTMBEP 

Decoder for the (18,6) Unit-Memory Code and the 

M=7, (3,1) convolutional Code when Feedback from 

Outer Decoders is Employed for a Simulated AWGN 

Channel with (E./N )_ at 1.25 dB and A=8. The 

b' o I 

Data are Taken from a Sample of 8000 Bytes. 


Erasure Error-Prob. of overall system 


Critei 

rion 

P 

q 

4 -error 
correcting 

6-error 

correcting 

8 -error 
correcting 

T=0.6 

0.00388 

0.01138 

1.891x10“^ 

6.387x10“^ 

9.416x10“^^ 

T=0.7 

0:00288 

0.01763 

2.498x10“^ 

7.122x10“^ 

8.207x10"^^ 

11 

o 

• 

CO 

0.00163 

0.02575 

3.725x10”^ 

8.138x10"® 

G. 173x10"^^ 

T=0.9 

0.00125 

0.03413 

1.178x10“^ 

3.554x10"^ 

3.283x10"^° 


Table 5.4: The performance of overall concatenated coding 

system employing a RTMBEP decoder, which is re- 
started when feedback from the outer decoder is ■ 
received, and a "erasures-and-rerrors" Reed-Solomon 
decoder for the (18,6) unit-memory code for a 

simulated AWGN channel with (E /N ) = 1.25 dB . 

■ b o I 



p 


BE 
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Concatenated with 

4-error-correcting 
RS code 

6-errorr-correcting 
RS code 

8-error-correcting 
RS code 
System I with 
M=6, (3.1) code 


System I with unit- 
memory code 


System 11 with unit- , 
memory code 

System III with M=7, 
(3.1) code 


System III with- 
unit-memory code 


System VI with M=7, 
(3,1) code (Zeoli) 


System IV with unit 
memory code 

System VI with unit- 
memory code 

System V with unit- 
memory code 


1.9 


2.Q 2.1 2.2 2.3 2.4 


2.5 2.-6 dB (E./N ) 

b' o o 


Figure 5.5: Performance of Concatenated Coding Systems 

Employing the Unit-Memory, (18.6) Convolutional 
are Compared to that of similar coding systems. 
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is consistent with the previous observation made about 
Figures 5.2 and 5.3. We can further make use of the reli- 
ability information put out by the RTMBEP decoder to erase 
some -less reliable bytes decoded by the inner decoder as 
we did in last section. The byte-error-probability , p, 
erasure probability/ q, and the over-all byte-error rate 
for the concatenated system are calculated and listed in 
Table 5.4 for thresholds equal to 0.6, 0.7, 0.8 and 0.9. 

The performance for T=0.8 is also plotted in Figure 5.5. 

Here, we observe nearly a 0.1 dB improvement, which is again 
consistent with the result of Section 5.1. 

5.3 ZEOLI’S CONCATENATED CODING SYSTEM AND ZEOLI'S MODI- 
FICATION ON THE UNIT-MEMORY CONVOLUTIONAL CODE. 

In [8] , Zeoli has proposed a concatenated coding 
system, which employs a rather long constraint’ length (K=32) 
convolutional code obtained by annexing a ’ long tail to the 
M=7, (3,1) convolutional code. While the state complexity 
of the Viterbi decoder remains the same as that for the M=7, 
(3,1) code, the annexed tail has absolutely no effect on the 
hard-decision decoding error-probability, until after an error 
has been made . But the tail provides excellent error-detec- 
tion once the Viterbi decoder starts to make mistakes. Since 
the decoder constraint length is much smaller than that of 
the encoder, the encoded branches in the Viterbi decoder 
assume the previous several hard-decisions as part of the 
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encoder state (See [8]) . The state metrics of the Viterbi 
decoder must become extremely ominous after a few branches 
once a decoding error -occurs. We are able to make use of 
this phenomenon to implement excellent erasure rules for 
the inner decoder. However, once the decoder makes one 
mistake, the decoder assumes an incorrect encoder state 
which leads to endless errors. The feedback from the outer 
decoder is, therefore, a necessity in order to reset the de- 
coder into correct state sequence and to terminate the error 
propagation. 

Motivated by the success of Zeoli's concatenated coding 
system, we annexed the unit-memory convolutional code with a 
three-branch— long "random tail” such that the resultant code 
is truely an M = 4, (18,6) convolutional code. The encoding 

matrices of this convolutional code are shown in Table 5.5. 

The length of the tail is chosen to be compatible in memory 
to that required for Zeoli's M = 31, (3,1) code. Since the 
decoder is still a Viterbi decoder for the (18,6) tinit-memory 
convolutional code, the decoder is going to make endless, mis- 
takes once an error occurs if the decision of the outer decod- 
er is not fed back to the Viterbi decoder; it makes no differ- 
ence whether the annexed M = 4, (18,6) code is non-catas trophic 
[12] or not. A software Viterbi decoder similar to the one 
described in last section is implemented to evaluate the per- 
formance of this system for the same AWGN channel. We chose 
the logarithm of the conditional probability P(s^|rj^ ) as 
the state metric for state s^. The reliability information 
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is derived from the difference of state matrices between 
states A+1 branches apart. The average of this quantity for 
correctly decoded state sequences as well as the standard de- 
viation of this quantity may then be calculated. The erasure 
rule is to erase the decoded byte when the metric difference' 
for A+1 brances is larger than T standard deviations away from 
the average. Because of the law of large numbers, we expect 
to erase very few correctly decoded bytes but to erase most of 
the incorrectly decoded bytes. The resultant byte-error- 
probability, p, and erasure probability, g, and the calculated 
byte-error-probability for the over-all concatenated coding 
system for T=1.5, T=1.8 and T=2.0, are listed in Table 5.6. 

I 

It is seen that the system performs best when the threshold 
is set at T=1.80. .If the metrics were Gaussian, as the central 
limit theorem v;ould suggest since the nximber of digit metrics 
added is large, this threshold of 1.80 correspond to an erasiire 
probability of 3.6% for correct bytes; we see actually that the 
fraction erased correct bytes is 2% somewhat less than. Gaussian 
estimate. We also include the performance of this system v;ith 
T=1.S in Figure 5.5. The performance of Zeoli’s originally 
orooosed system is directly taken from [8] . 'We observe about 
0.2 d3 improvement over Viterbi decoding with feedback for the 
unit-memory code in the last section compared to the same 

system vjith Zeoli's modification. This improvement is 
again consistent with that observed from the Viterbi decoding 

with feedback for the M=7, (3,1) code compared to Zeoli’s 
original system which makes us confident of our use of Zeoli's 
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111000 

110100 

110000 


000011 

000111 

001011 

011100 

011010 

011000 


000110 

001110 

010110 

001110 

001101 

001100 


001100 

oliloo 

101100 

000111 

100110 

000110 

G 

1 ~ 011000 

111000 

011001 

100011 

010011 

000011 


110000 

110001 

110010 

110001 

101001 

100001 


100001 

100011 

100101 

000110 

000001 

101111 


011000 

111001 

011000 

100011 

000011 

010011 


110001 

110010 

110000 

110001 

100110 

100001 


100011 

100101 

100001 

111000 

110101 

001000 

G 

3 000111 

000011 

001011 

011000 

011010 

011100 


000110 

001110- 

010110 

001100 

011100 

110110 


001100 

011100 

101100 



mill 

010100 

000000 





000111 

111010 

100000 





000000 

111111 

010100 





100000 

000111 

111010 




4 

010100 

000000 

mill 





111010 

100000 

000111 




Table 5.5: The encoding matrices of a M=4, (18,6) convolu- 

tional code obtained by anhexing a random tail 
to the (18,6) unit-memory convolutional code 


Byte-Error-Probability of Over-all 
Concatenated Coding System 


T 

p 

q 

4 -error 
correcting 

6 -error 
correcting 

8-error 

correcting 

1.50 

0.00125 

0.03788 

2.095x10“^ 

8.175x10'"^ 

9.465x10“^° 

1.80 

0.00263 

0,02088 

3.602xl0."’^ 

1.074x10"^ 

1.245x10“^° 

2.00 

0.00425 

0.01450 

4.168x10“^ 

1.899x10"*^ 

3.708x10"^° 


Table 5.6: The performance of Zeoli's type of concatenated 

coding system employing the M=4, (18,6) convolu- 
tional code and a Viterbi decoder with feedback 
for the (18,6) unit-memory code for different 
erasing thresholds for .a simulated AWGN channel 
at 1.25 dB. The data are taken from 

a sample of 8000 bytes, and the decoding delay 
for the Viterbi decoder is 8. 
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data v;ithout checking it by our own simulations. 

We also point out that our use of the "normalized” 
thresholds in terms of mean and standard deviations of the 
increment of branch metric is much more convenient and makes 
much more sense as an erasure criterion than Zeoli’s choice 
of the state metric itself, since our choice of the thresholds 
are independent of the particular metric which the Viterbi 
decoder employs . 
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DEGRADATION OF PERFORMANCE FQR EMPLOYING HIGHER RATE 
INNER CODES. 

We have extensively studied block-convolutional 
concatenated coding systems employing rate 1/3 convolutional 

g 

codes and Reed-Solomon codes over GF (2 ) . However, it is 
sometimes desired in practice to operate the inner convolu- 
tional codes at a higher rate, rate 1/2 in particular, in 
order to ease the burden imposed on the phase locked loops 
in the receiver. We shall illustrate a heuristic approach 
to estimate the performance of similar concatenated coding 
systems with rate 1/2 coding systems. 

From past experience, the performance of rate 1/2 
convolutional coding system is about 0.5 dB inferior to 
that of rate 1/3 convolutional coding system of the same 
complexity. We therefore tested the performance of a soft- 
ware Viterbi decoder (without feedback) for the M=6; (2.1) 
convolutional code in an AWGN channel at I ~ dB 

or, equivalently, E^/N^ = -1.25 dB. The results of this 
simulation and the calculated overall byte-error-probabi- 
lity when it is concatenated with the Reed-SOlomon codes 
are listed in Table 5.7. Comparing the results of Table 
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By te-Error-Prob . 
of the Viterbi 
Decoder' 


0.0305 


Byte-Error-Probability of the Over-all 
Concatenated Coding System 
4-error- 6 -error- 8 -error- 10 -error- 
correcting correcting correcting correcting 

f 

6.168x10"^ 6.287x10”^ 3.285xl0“^ l.OOexlo"^ 


Table 5.7: The Performance of Viterbi Decoder for a 14=67 (2,1) 

Convolutional ' Code in a Simulated AWGN Channel at 

(E./N )_ = 1.75 dB. The data are taken from a 
b o X 

sample of 400 bytes and the decoding delay is 48 
branches . 


5.7 To those given in Table -2.3 for M= 6 / {3.1) code and the 
(18,6) unit-memory code, we find that the performance of rate 
1/2 code operated at j = 1-75 dB is almost equivalent 

to that of the unit memory code operated at (Ej^/N^) ^ = 1.0 dB 
or that of the M= 6 , (3,1) code operated at j equal to 

somewhere above 1.25 dB. This is consistent with the past 
experience. The performance of the concatenated coding 
systems using the three codes are plotted in Figure 5.6. Once 
again the difference between the M= 6 , (3,1) coding system and 
the M= 6 , (2.1) is about 0.5 dB. This is obvious because 
for the two coding systems to have the same error probability, 
it is necessary that the two inner decoders have exactly 
the same error-probability. 

Unfortunately, the (12,6) unit-memory convolutional 
code has exactly the same free distance as the M= 6 , ( 2 , 1 ) code 
(See Table 2.1) . Therefore, there is no distance advantage in 
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Figure 5 , 6 : Performance of the concatenated system I described in section 5 . 1 . 

The rate 1/2 system-is operated at (Ev/N )y = 1.75 dB, whereas' the rate 
1/3 systems are operated at (Ej^/N^)j = l? 2 b dB. 
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utilizing the unit-memory convolutional code. 

In conclusion, a penalty of approximately 0.5 dB 
is paid when a (3,1) convolutional code is replaced by a (2,1) 
convolutional code of the same state-complexity; and a unit- 
memory code which has a free distance one greater than an 
(n^, 1) convolutional code of the same state-complexity has 
0.3 dB advantage compared to this (n^ , 1) convolutional code . 

In the case we studied, a sum of about 0.8 dB is sacrified when 
we choose a rate 1/2 convolutional code to replace the rate 
1/3 unit-memory code. In the case where the byte-size is 5 
bits, from Table 2.1, we see the free distances of (15,5) 
and (10,5) unit-memory codes are 15 and 9 which are 2 greater 
and 1 greater than the (3,1) and (2,1) codes of the same 
complexity; we expect again an approximately 0.8 dB loss when 
we choose the (10,5) code in place of the (15,5) unit memory 
code. But, when the byte-size is 4, on the other hand, we 
see that there is no distance advantage for the (12,4) unit- 
memory code compared to the (3,1) code; while, on the contrary, 
the free distance of the (8^4) unit-memory code is one greater 
than the (2,1) code; therefore the 0.5 dB loss due to the 
increase of rate should be somewhat compensated by the better 
free distance of the (8,4) unit-memory convolutional code. 
Therefore, we might expect only a 0.2 dB degradation. 



5.5. 


CONFIDENCE INTERVALS FOR THE SIMULATION RESULTS 
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The performance of various block-convolutional concatenated 
coding systems over a simulated ANGN channel was studied and com- 
pared in the previous sections. The fact that the overall byte- 
error rate is calculated from the byte-error rate of the inner 
decoders makes it possible to carry out the simulations with a 
moderate size of samples. Assuming that the decoder makes a error 
with probability for each byte-decision, the number of byte 

errors within n byte-decisions is a binomial random variable with 
parameters n and Pgg* Then, the mean value of this random variable, 
is, nPgj., and the standard deviation is /nP^^ (l-P^^) . If n is 
sufficiently large, this binomial random variable can be approximated 
by a Gaussian random variable with the same mean and variance. 
since'95% of the samples of a Gaussian random variable are within the 
interval specified by the mean and twice of the standard deviation, 
we are confident that with more than 95% probability, the actual byte- 
error rats for the inner decoder is in the interval specified by 
-2 >''nP^^Tl-P^P^, ■ Pb 2 +2 /nPgg ). These 95% confidence 

intervals are indicated in Table 2,3. 

•The performances of System I using the M=6, (3.1) code and 
M=l, (18.6) unit-memory code corresponding the upper and lower limits 
of these intervals are calculated and shown in Figure 5.7. We conclude 
with 95% confidence that the actual performance of the concatenated 
coding system deviates no more than 0.1 dB from our simulation results 
Moreover, since all the simulation results are obtained through the 
same pseudo-random mmber sequence, the relative difference in per- 
formance among various systems are, in fact, much more acturate than 
the 0.1 dB confidence interval obse 3 Tved here. 
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■ 1.9 2.0 2.1 2.2 2.3 2.'4 2.5 2.6 

Figure 5.?. The 95% Confidence Intervals for the Performance 
of System I with M=6, (3-1) Convolutional Code 

and (18.6) Unit-Memory Code, 



CHAPTER VI 


SUMMARY AND CONCLUSIONS 

In Chapter Y, we have extensively studied block- 
convolutional concatenated coding systems with various modi- 
fications. It appears that the advantage of employing unit- 
memory convolutional codes can improve the performance nearly 
0.3 dB. The feedback from the outer decoder to restart the 
Viterbi decoder also contributes about 0.3 dB, But, surprising 
ly, the feedback from the outer decoder to restart the RTMBEP 
decoder offers approximately 0.5 dB advantage, which is’ 0.2 dB 
more than the same feedback is able to help the Viterbi decoder 
As a result, this might be the principal occasion where the 
use of RTMBEP decoding is justified. Another unexpected re- 
sult is that soft-decisions by the inner decoder in conjunction 
with an erasures-and-errors outer decoder only improves the 
overall performance by about 0.05 dB to 0.1 dB for RTMBEP 
decoding. Even with Zeoli's modification which provides the 
best error detection capability, soft-decisions in conjunction 
with an erasures-and-errors outer decoder can improve the 
performance by only approximately 0.2 dB. We summarize the 
effects of each feature discussed above on the performance 
of the block-convolutional concatenated coding system in 
Figure 6.1. The figure is drawn in terms of a dB scale. As 
a communications engineer starts to choose a coding system, 
the first question he faces is whether his phase-locked-loop 
can tolerate the burden of a rate 1/3 coding system, if the 
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answer is positive, he gains 0,5 dB. Then, he is to decide 
which inner code to employ^ to choose the M=7, (3,1) code gives 
0.2 dB advantage over the M=6, (3,1) code but twice the number 
of states in the decoder is required; whereas to choose the 
M=l, (18,6) code has 0.3 dB advantage with the number of states 
required kept the same, but more branch connections are required. 
The third question is whether he allows the decisions of the 
outer decoder to feedback to the inner decoder; if not the 
obvious choice is Viterbi decoding; otherwise, • he can gain 
0.3 dB or 0.5 dB depends on whether the Viterbi decoder or 
the RTMBEP decoder is utilized. And finally, if soft-decisions 
are desired, he can gain 0.2 dB through Zeoli's type of erasure 
scheme if he use a Viterbi decoder, or gains only 0.05 dB if 
the plain RTMBEP erasure scheme is employed. 

As we observed from Figure 5.5, the leading contenders 
for. a good concatenated system are the sophisticated schemes 
of (1) Zeoli's modification- with the unit-memory code, (2) 
hard-decision or (3) the soft-decision RTMBEP decoding with 
feedback from the outer decoder for the unit-memory code. 

Among them, the soft-decision RTMBEP decoder with feedback 
performs the best. In terms of hardware implementation, 

Zeoli’s modification with the unit-memory code and the 
hard-decision RTMBEP decoder are approximately of the same 
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complexity. However, since the operation of the Viterbi decoder for 
the Zeoli's system depends on the correct feedback from the outer 
decoder, there is always a slim chance that the outer decoder fails 
to provide correct decisions to the Viterbi decoder. Since the 
encoder constraint length is much larger than the decoder constraint 
length, this can cause endless errors’ as if a catastrophic convolutional 
code were used. Thus, it is necessary to send synchronization signals 
periodically to reset the Viterbi decoder to guarantee restoration of ■ 
normal operation. The RTMBEP decoder has the same constraint length 
as that of the encoder, therefore, the decoder is able to recover 
from errors in a few branches by itself without feedback. The feedback 
from the outer decoder only speeds this process up; therefore, when 
an error is fed back, the most damage it can cause is for. the RTMBEP 
decoder to make a few more errors before it recovers by itself. This 
is certainly a very desirable advantage for a concatenated coding 
system. 

Moreover, because the decoder can restore its normal operation 
quickly, the degree of interleaving required for this scheme is con- 
siderably less than the Reed-Solomon ’ block length required for the 
Zeoli's scheme. 

As microprocessors are mass-produced, we foresee this as a pro- 
mising practical scheme to achieve very reliable communication on a 
very noisy channel. Moreover, in our simulations. We have studied 
only the case when the outer decoder sends a corrected feedback to 
the inner decoder when an error was detected by the outer decoder. 

It is conceivable that one might improve the performance of the 
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concatenated coding system by employing the feedback from the 
outer decoder whenever the decision of the inner decoder is erased. 

This scheme requires further simulation results for different 
thresholds to determine its performance, but it is certainly a 
promising direction for future research. 

Finally, as an interesting reminder to information theorists, 
we note that for the RTMBEP decoder with feedback system employing 
the unit-memory code concatenated with the .(63 53) , 6 -error-correcting 
RS code, we can achieve a byte-error-probability of 10 at E_/N =-3.25 
dB (or(Ej^/N^)^ = 1.25 dB) . The cut-off rate, R^, of this 8-level 
quantized AWGN channel is 0.275, and the channel capacity is 0.44 
-whereas the over-all rate of our concatenated coding system is 0.27. 

It seems that the cut-off rate is still- the practical limit of rate 
for reliable communications, even for a very sophisticated con- 
catenated coding system. 
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